[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:37:11 dist-test-slave-4tdj krb5kdc[31994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:37:11 dist-test-slave-4tdj krb5kdc[31994](info): set up 2 sockets
Aug 10 06:37:11 dist-test-slave-4tdj krb5kdc[31994](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 06:37:13.053009 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:37:13 dist-test-slave-4tdj krb5kdc[31994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:37:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:37:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:37:13.225250 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:35321
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:35321
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:13.683255 32031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:13.683663 32031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:13.683765 32031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:13.691311 32031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:37:13.691461 32031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:13.691536 32031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:13.691596 32031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:37:13.691640 32031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:37:13.702346 32031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:35321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:35321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:13.704174 32031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:13.706149 32031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:13.725322 32040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:13.725757 32046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:13.726011 32043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:13.726958 32031 server_base.cc:1034] running on GCE node
I20260810 06:37:13.728121 32031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:13.730391 32031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:13.731617 32031 hybrid_clock.cc:648] HybridClock initialized: now 1786343833731542 us; error 96 us; skew 500 ppm
I20260810 06:37:13.732157 32031 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:13 dist-test-slave-4tdj krb5kdc[31994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343833, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:13.735622 32031 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:37:13.738152 32031 webserver.cc:460] Webserver started at http://127.31.59.254:41419/ using document root <none> and password file <none>
I20260810 06:37:13.739375 32031 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:13.739624 32031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:13.740137 32031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:13.744208 32031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "a58afd2f0e384b048a91596d3789db80"
format_stamp: "Formatted at 2026-08-10 06:37:13 on dist-test-slave-4tdj"
server_key: "59e215831d1659d2d7a1909eb87b8cfd"
server_key_iv: "07c910f3cd3b3d20a422893425553381"
server_key_version: "encryptionkey@0"
I20260810 06:37:13.745039 32031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "a58afd2f0e384b048a91596d3789db80"
format_stamp: "Formatted at 2026-08-10 06:37:13 on dist-test-slave-4tdj"
server_key: "59e215831d1659d2d7a1909eb87b8cfd"
server_key_iv: "07c910f3cd3b3d20a422893425553381"
server_key_version: "encryptionkey@0"
I20260810 06:37:13.753470 32031 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260810 06:37:13.759712 32055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:13.761754 32031 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:37:13.762055 32031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "a58afd2f0e384b048a91596d3789db80"
format_stamp: "Formatted at 2026-08-10 06:37:13 on dist-test-slave-4tdj"
server_key: "59e215831d1659d2d7a1909eb87b8cfd"
server_key_iv: "07c910f3cd3b3d20a422893425553381"
server_key_version: "encryptionkey@0"
I20260810 06:37:13.762367 32031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:13.779541 32031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:13.784582 32031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:13.785001 32031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:13.813905 32031 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:35321
I20260810 06:37:13.813845 32134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:35321 every 8 connection(s)
I20260810 06:37:13.816534 32031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:37:13.824326 32135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:13.824352 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 32031
I20260810 06:37:13.824795 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:37:13.827785 31983 external_mini_cluster.cc:1428] Setting key 73c83fa9373c73f8fd8bbab49251a6d7
I20260810 06:37:13.843335 32135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80: Bootstrap starting.
I20260810 06:37:13.849874 32135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:13.852437 32135 log.cc:824] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:13.857318 32135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80: No bootstrap required, opened a new log
Aug 10 06:37:13 dist-test-slave-4tdj krb5kdc[31994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:13.869540 32135 raft_consensus.cc:348] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58afd2f0e384b048a91596d3789db80" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35321 } }
I20260810 06:37:13.870185 32135 raft_consensus.cc:374] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:13.870414 32135 raft_consensus.cc:729] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a58afd2f0e384b048a91596d3789db80, State: Initialized, Role: FOLLOWER
I20260810 06:37:13.871842 32135 consensus_queue.cc:260] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58afd2f0e384b048a91596d3789db80" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35321 } }
I20260810 06:37:13.872292 32135 raft_consensus.cc:388] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:37:13.872623 32135 raft_consensus.cc:482] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:37:13.872861 32135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:13.876595 32135 raft_consensus.cc:504] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58afd2f0e384b048a91596d3789db80" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35321 } }
I20260810 06:37:13.877840 32135 leader_election.cc:302] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a58afd2f0e384b048a91596d3789db80; no voters:
I20260810 06:37:13.879006 32135 leader_election.cc:288] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:37:13.879205 32144 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:13.879473 32138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:13.834953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49108 (local address 127.31.59.254:35321)
0810 06:37:13.836177 (+ 1224us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:13.836194 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:13.839547 (+ 3353us) server_negotiation.cc:407] Connection header received
0810 06:37:13.851386 (+ 11839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:13.851445 (+ 59us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:13.852128 (+ 683us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:13.852619 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:13.856685 (+ 4066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:13.859082 (+ 2397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:13.860779 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:13.862027 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:13.862287 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:13.868536 (+ 6249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:13.868636 (+ 100us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:13.868683 (+ 47us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:13.868729 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:13.872926 (+ 4197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:13.873722 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:13.873730 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:13.873738 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:13.873982 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:13.874466 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:13.874471 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:13.874474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:13.875452 (+ 978us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:13.875980 (+ 528us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:13.876913 (+ 933us) server_negotiation.cc:299] Negotiation successful
0810 06:37:13.877331 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":767,"thread_start_us":363,"threads_started":1}
I20260810 06:37:13.880055 32144 raft_consensus.cc:686] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80 [term 1 LEADER]: Becoming Leader. State: Replica: a58afd2f0e384b048a91596d3789db80, State: Running, Role: LEADER
I20260810 06:37:13.880955 32144 consensus_queue.cc:237] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58afd2f0e384b048a91596d3789db80" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35321 } }
I20260810 06:37:13.887392 32144 sys_catalog.cc:455] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a58afd2f0e384b048a91596d3789db80. Latest consensus state: current_term: 1 leader_uuid: "a58afd2f0e384b048a91596d3789db80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58afd2f0e384b048a91596d3789db80" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35321 } } }
I20260810 06:37:13.887957 32144 sys_catalog.cc:458] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:13.888123 32147 sys_catalog.cc:455] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a58afd2f0e384b048a91596d3789db80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58afd2f0e384b048a91596d3789db80" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35321 } } }
I20260810 06:37:13.888748 32147 sys_catalog.cc:458] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:13.891383 32135 sys_catalog.cc:564] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:37:13.898428 32150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:37:13.907032 32150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:37:13.921934 32150 catalog_manager.cc:1347] Generated new cluster ID: c1c973e4c1d344bca87415dfd0329946
I20260810 06:37:13.922044 32150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:37:13.963718 32150 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:37:13.965263 32150 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:37:13.973297 32150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a58afd2f0e384b048a91596d3789db80: Generated new TSK 0
I20260810 06:37:13.974561 32150 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
I20260810 06:37:14.028072 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:35321
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:14.544404 32178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:14.544698 32178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:14.544844 32178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:14.553478 32178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:14.553599 32178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:14.553763 32178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:37:14.564756 32178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:35321
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:14.566049 32178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:14.567806 32178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:14.585121 32188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:14.585095 32191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:14.585583 32196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:14.586410 32178 server_base.cc:1034] running on GCE node
I20260810 06:37:14.587007 32178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:14.588160 32178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:14.589414 32178 hybrid_clock.cc:648] HybridClock initialized: now 1786343834589319 us; error 40 us; skew 500 ppm
I20260810 06:37:14.589767 32178 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:14 dist-test-slave-4tdj krb5kdc[31994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343834, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:14.594372 32178 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:37:14.596091 32178 webserver.cc:460] Webserver started at http://127.31.59.193:34465/ using document root <none> and password file <none>
I20260810 06:37:14.596788 32178 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:14.596994 32178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:14.597337 32178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:14.600811 32178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "c92a1e0fc77e44c5aff7dd30aa90ce97"
format_stamp: "Formatted at 2026-08-10 06:37:14 on dist-test-slave-4tdj"
server_key: "5911321862f85e5fc6e1943f4a2112fd"
server_key_iv: "6f1b841683adf380b561212c707c0eae"
server_key_version: "encryptionkey@0"
I20260810 06:37:14.601758 32178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "c92a1e0fc77e44c5aff7dd30aa90ce97"
format_stamp: "Formatted at 2026-08-10 06:37:14 on dist-test-slave-4tdj"
server_key: "5911321862f85e5fc6e1943f4a2112fd"
server_key_iv: "6f1b841683adf380b561212c707c0eae"
server_key_version: "encryptionkey@0"
I20260810 06:37:14.608201 32178 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260810 06:37:14.612696 32211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:14.614012 32178 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:37:14.614182 32178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "c92a1e0fc77e44c5aff7dd30aa90ce97"
format_stamp: "Formatted at 2026-08-10 06:37:14 on dist-test-slave-4tdj"
server_key: "5911321862f85e5fc6e1943f4a2112fd"
server_key_iv: "6f1b841683adf380b561212c707c0eae"
server_key_version: "encryptionkey@0"
I20260810 06:37:14.614454 32178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:14.642518 32178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:14.648490 32178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:14.648923 32178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:14.650472 32178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:14.652902 32178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:14.653023 32178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:14.653272 32178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:14.653410 32178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:14.717267 32178 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:46689
I20260810 06:37:14.717427 32405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:46689 every 8 connection(s)
I20260810 06:37:14.719089 32178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:37:14.727351 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 32178
I20260810 06:37:14.727761 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:37:14.728506 31983 external_mini_cluster.cc:1428] Setting key 733b183248d27475eccbbe15600b38d7
Aug 10 06:37:14 dist-test-slave-4tdj krb5kdc[31994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343834, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:14.744549 32409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:14.722517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:56973 (local address 127.31.59.254:35321)
0810 06:37:14.723147 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:14.723153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:14.724432 (+ 1279us) server_negotiation.cc:407] Connection header received
0810 06:37:14.727263 (+ 2831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:14.727271 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:14.727360 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:14.727485 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:14.729179 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:14.730386 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:14.732696 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:14.733310 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:14.733439 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:14.737828 (+ 4389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:14.737832 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:14.737835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:14.737867 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:14.740849 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:14.741887 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:14.741890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:14.741893 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:14.742066 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:14.742650 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:14.742653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:14.742655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:14.742990 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:14.743121 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:14.744047 (+ 926us) server_negotiation.cc:299] Negotiation successful
0810 06:37:14.744239 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":318,"threads_started":1}
I20260810 06:37:14.746021 32410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:14.723175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:35321 (local address 127.31.59.193:56973)
0810 06:37:14.724274 (+ 1099us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:14.724317 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:14.726957 (+ 2640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:14.727753 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:14.727761 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:14.728337 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:14.728987 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:14.729007 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:14.730576 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:14.730581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:14.732544 (+ 1963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:14.732563 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:14.733501 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:14.733528 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:14.733822 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:14.734565 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:14.734588 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:14.737608 (+ 3020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:14.740987 (+ 3379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:14.740996 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:14.741012 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:14.741773 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:14.742172 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:14.742177 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:14.742181 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:14.742550 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:14.743137 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:14.743145 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:14.743606 (+ 461us) client_negotiation.cc:769] Sending connection context
0810 06:37:14.743975 (+ 369us) client_negotiation.cc:240] Negotiation successful
0810 06:37:14.744504 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"thread_start_us":438,"threads_started":1}
WARNING: no policy specified for kudu/127.31.59.194@KRBTEST.COM; defaulting to no policy
I20260810 06:37:14.747645 32406 heartbeater.cc:347] Connected to a master server at 127.31.59.254:35321
I20260810 06:37:14.748117 32406 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:14.749191 32406 heartbeater.cc:511] Master 127.31.59.254:35321 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.194@KRBTEST.COM" created.
I20260810 06:37:14.753221 32079 ts_manager.cc:194] Registered new tserver with Master: c92a1e0fc77e44c5aff7dd30aa90ce97 (127.31.59.193:46689)
I20260810 06:37:14.755491 32079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:56973
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
I20260810 06:37:14.771637 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:35321
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:15.186057 32414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:15.186422 32414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:15.186558 32414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:15.194964 32414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:15.195096 32414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:15.195307 32414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:37:15.205302 32414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:35321
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:15.206575 32414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:15.207864 32414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:15.221396 32425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:15.221762 32426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:15.222036 32430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:15.223080 32414 server_base.cc:1034] running on GCE node
I20260810 06:37:15.223732 32414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:15.224617 32414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:15.225862 32414 hybrid_clock.cc:648] HybridClock initialized: now 1786343835225770 us; error 43 us; skew 500 ppm
I20260810 06:37:15.226207 32414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:15 dist-test-slave-4tdj krb5kdc[31994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343835, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:15.229466 32414 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:37:15.230820 32414 webserver.cc:460] Webserver started at http://127.31.59.194:33867/ using document root <none> and password file <none>
I20260810 06:37:15.231441 32414 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:15.231563 32414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:15.231905 32414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:15.234869 32414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "35b67d0c45724bfebe575897dd80952f"
format_stamp: "Formatted at 2026-08-10 06:37:15 on dist-test-slave-4tdj"
server_key: "7f0d171c55cb8511f23a010eb07b56ab"
server_key_iv: "c6c46e8b963c6da829ba13a8b7e37cda"
server_key_version: "encryptionkey@0"
I20260810 06:37:15.235575 32414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "35b67d0c45724bfebe575897dd80952f"
format_stamp: "Formatted at 2026-08-10 06:37:15 on dist-test-slave-4tdj"
server_key: "7f0d171c55cb8511f23a010eb07b56ab"
server_key_iv: "c6c46e8b963c6da829ba13a8b7e37cda"
server_key_version: "encryptionkey@0"
I20260810 06:37:15.240854 32414 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:37:15.244161 32455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:15.245464 32414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:37:15.245651 32414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "35b67d0c45724bfebe575897dd80952f"
format_stamp: "Formatted at 2026-08-10 06:37:15 on dist-test-slave-4tdj"
server_key: "7f0d171c55cb8511f23a010eb07b56ab"
server_key_iv: "c6c46e8b963c6da829ba13a8b7e37cda"
server_key_version: "encryptionkey@0"
I20260810 06:37:15.245877 32414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:15.279821 32414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:15.284078 32414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:15.284379 32414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:15.285624 32414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:15.287197 32414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:15.287276 32414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:15.287405 32414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:15.287458 32414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:15.329789 32414 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:44031
I20260810 06:37:15.329872 32662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:44031 every 8 connection(s)
I20260810 06:37:15.331101 32414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:37:15.339052 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 32414
I20260810 06:37:15.339291 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:37:15.339952 31983 external_mini_cluster.cc:1428] Setting key 55273d367fe1af3bd8102b249a517c81
Aug 10 06:37:15 dist-test-slave-4tdj krb5kdc[31994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343835, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:15.355095 32666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:15.333966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:56335 (local address 127.31.59.254:35321)
0810 06:37:15.334549 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:15.334555 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:15.335464 (+ 909us) server_negotiation.cc:407] Connection header received
0810 06:37:15.337417 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:15.337422 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:15.337489 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:15.337582 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:15.339332 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:15.340478 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:15.342253 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:15.342722 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:15.342818 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:15.347304 (+ 4486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:15.347310 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:15.347314 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:15.347350 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:15.350740 (+ 3390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:15.351752 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:15.351757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:15.351760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:15.351921 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:15.352502 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:15.352507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:15.352511 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:15.352924 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:15.353095 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:15.354504 (+ 1409us) server_negotiation.cc:299] Negotiation successful
0810 06:37:15.354730 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":257,"threads_started":1}
I20260810 06:37:15.355721 32670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:15.334619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:35321 (local address 127.31.59.194:56335)
0810 06:37:15.335312 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:15.335349 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:15.337176 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:15.337769 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:15.337776 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:15.338499 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:15.339137 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:15.339155 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:15.340658 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:15.340661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:15.342116 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:15.342132 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:15.342937 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:15.342944 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:15.343223 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:15.344006 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:15.344064 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:15.347068 (+ 3004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:15.350913 (+ 3845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:15.350922 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:15.350938 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:15.351568 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:15.352074 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:15.352079 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:15.352083 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:15.352324 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:15.353096 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:15.353104 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:15.353540 (+ 436us) client_negotiation.cc:769] Sending connection context
0810 06:37:15.353843 (+ 303us) client_negotiation.cc:240] Negotiation successful
0810 06:37:15.354317 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":284,"threads_started":1}
I20260810 06:37:15.357327 32663 heartbeater.cc:347] Connected to a master server at 127.31.59.254:35321
I20260810 06:37:15.357633 32663 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:15.358588 32663 heartbeater.cc:511] Master 127.31.59.254:35321 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.59.195@KRBTEST.COM; defaulting to no policy
I20260810 06:37:15.360159 32079 ts_manager.cc:194] Registered new tserver with Master: 35b67d0c45724bfebe575897dd80952f (127.31.59.194:44031)
I20260810 06:37:15.360966 32079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:56335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
I20260810 06:37:15.381332 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:35321
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:37:15.759388 32406 heartbeater.cc:503] Master 127.31.59.254:35321 was elected leader, sending a full tablet report...
W20260810 06:37:15.774147 32677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:15.774448 32677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:15.774617 32677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:15.782584 32677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:15.782701 32677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:15.782867 32677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:37:15.793682 32677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:35321
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:15.794958 32677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:15.796348 32677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:15.809283 32688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:15.810038 32689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:15.810408 32692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:15.811101 32677 server_base.cc:1034] running on GCE node
I20260810 06:37:15.811659 32677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:15.812500 32677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:15.813717 32677 hybrid_clock.cc:648] HybridClock initialized: now 1786343835813660 us; error 44 us; skew 500 ppm
I20260810 06:37:15.814122 32677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:15 dist-test-slave-4tdj krb5kdc[31994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343835, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:15.818599 32677 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:37:15.820423 32677 webserver.cc:460] Webserver started at http://127.31.59.195:37679/ using document root <none> and password file <none>
I20260810 06:37:15.821542 32677 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:15.821672 32677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:15.821985 32677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:15.825454 32677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "b3b9cc6fa1d3495aa13301ff23716e9a"
format_stamp: "Formatted at 2026-08-10 06:37:15 on dist-test-slave-4tdj"
server_key: "8d286fffaf5fb40d037c53fb48aef31d"
server_key_iv: "e59853dc9c34a7c3c1ea6e91e7485bde"
server_key_version: "encryptionkey@0"
I20260810 06:37:15.826207 32677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "b3b9cc6fa1d3495aa13301ff23716e9a"
format_stamp: "Formatted at 2026-08-10 06:37:15 on dist-test-slave-4tdj"
server_key: "8d286fffaf5fb40d037c53fb48aef31d"
server_key_iv: "e59853dc9c34a7c3c1ea6e91e7485bde"
server_key_version: "encryptionkey@0"
I20260810 06:37:15.832250 32677 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260810 06:37:15.836374 32708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:15.837725 32677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:37:15.837865 32677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "b3b9cc6fa1d3495aa13301ff23716e9a"
format_stamp: "Formatted at 2026-08-10 06:37:15 on dist-test-slave-4tdj"
server_key: "8d286fffaf5fb40d037c53fb48aef31d"
server_key_iv: "e59853dc9c34a7c3c1ea6e91e7485bde"
server_key_version: "encryptionkey@0"
I20260810 06:37:15.838109 32677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:15.854698 32677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:15.863020 32677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:15.863265 32677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:15.864348 32677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:15.866004 32677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:15.866150 32677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:15.866381 32677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:15.866520 32677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260810 06:37:15.922075 32677 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:44633
I20260810 06:37:15.922116 440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:44633 every 8 connection(s)
I20260810 06:37:15.923525 32677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:37:15.927985 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 32677
I20260810 06:37:15.928216 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:37:15.928799 31983 external_mini_cluster.cc:1428] Setting key a70245d585759e27295679d16284d937
Aug 10 06:37:15 dist-test-slave-4tdj krb5kdc[31994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343835, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:15.946278 445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:15.926489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:34161 (local address 127.31.59.254:35321)
0810 06:37:15.926927 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:15.926932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:15.928370 (+ 1438us) server_negotiation.cc:407] Connection header received
0810 06:37:15.930891 (+ 2521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:15.930896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:15.930988 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:15.931236 (+ 248us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:15.933205 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:15.934133 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:15.935722 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:15.936150 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:15.936257 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:15.940112 (+ 3855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:15.940118 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:15.940121 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:15.940154 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:15.943098 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:15.943799 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:15.943803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:15.943805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:15.943922 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:15.944473 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:15.944478 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:15.944481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:15.944804 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:15.944917 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:15.945771 (+ 854us) server_negotiation.cc:299] Negotiation successful
0810 06:37:15.945953 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":237,"threads_started":1}
I20260810 06:37:15.947438 446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:15.927059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:35321 (local address 127.31.59.195:34161)
0810 06:37:15.927943 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:15.928006 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:15.930618 (+ 2612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:15.931482 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:15.931490 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:15.932209 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:15.933037 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:15.933058 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:15.934283 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:15.934287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:15.935546 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:15.935559 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:15.936355 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:15.936360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:15.936556 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:15.937225 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:15.937247 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:15.939888 (+ 2641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:15.943206 (+ 3318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:15.943212 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:15.943224 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:15.943695 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:15.943995 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:15.943997 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:15.944000 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:15.944348 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:15.944915 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:15.944922 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:15.945386 (+ 464us) client_negotiation.cc:769] Sending connection context
0810 06:37:15.945654 (+ 268us) client_negotiation.cc:240] Negotiation successful
0810 06:37:15.946240 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":368,"threads_started":1}
I20260810 06:37:15.948819 441 heartbeater.cc:347] Connected to a master server at 127.31.59.254:35321
I20260810 06:37:15.949127 441 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:15.949846 441 heartbeater.cc:511] Master 127.31.59.254:35321 requested a full tablet report, sending...
I20260810 06:37:15.951202 32079 ts_manager.cc:194] Registered new tserver with Master: b3b9cc6fa1d3495aa13301ff23716e9a (127.31.59.195:44633)
I20260810 06:37:15.951972 32079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:34161
I20260810 06:37:15.958472 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 06:37:15 dist-test-slave-4tdj krb5kdc[31994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343835, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:37:16 dist-test-slave-4tdj krb5kdc[31994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343835, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:37:16.010910 452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:15.992326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58012 (local address 127.31.59.193:46689)
0810 06:37:15.993080 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:15.993091 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:15.993127 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:37:15.993307 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:15.993316 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:15.993525 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:15.993697 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:15.995823 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:15.997022 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:15.999079 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:15.999830 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:15.999955 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:16.004829 (+ 4874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:16.004841 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:16.004856 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:16.004923 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:16.008164 (+ 3241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:16.008812 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:16.008819 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:16.008822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:16.008953 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:16.009332 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:16.009336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:16.009338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:16.009685 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:16.009967 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:16.010320 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 06:37:16.010579 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":385,"threads_started":1}
W20260810 06:37:16.012298 32256 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58012
Aug 10 06:37:16 dist-test-slave-4tdj krb5kdc[31994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:37:16 dist-test-slave-4tdj krb5kdc[31994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:37:16.057654 452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:16.043654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58020 (local address 127.31.59.193:46689)
0810 06:37:16.043868 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:16.043874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:16.044130 (+ 256us) server_negotiation.cc:407] Connection header received
0810 06:37:16.044310 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:16.044315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:16.044417 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:16.044506 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:16.046119 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:16.046861 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:16.048115 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:16.048540 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:16.048657 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:16.052724 (+ 4067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:16.052729 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:16.052733 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:16.052769 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:16.055328 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:16.056052 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:16.056055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:16.056058 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:16.056192 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:16.056547 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:16.056551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:16.056553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:16.056781 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:16.056952 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:16.057206 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 06:37:16.057368 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 06:37:16.059602 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 32178
I20260810 06:37:16.072135 452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:16.060653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58024 (local address 127.31.59.193:46689)
0810 06:37:16.060851 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:16.060856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:16.061085 (+ 229us) server_negotiation.cc:407] Connection header received
0810 06:37:16.061193 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:16.061197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:16.061249 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:16.061318 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:16.062402 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:16.063083 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:16.064352 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:16.064804 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:16.064894 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:16.067345 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:16.067350 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:16.067353 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:16.067381 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:16.070033 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:16.070636 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:16.070640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:16.070642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:16.070764 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:16.071037 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:16.071040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:16.071042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:16.071256 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:16.071429 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:16.071672 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 06:37:16.071813 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:37:16.072961 32394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:16.161460 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 32178
I20260810 06:37:16.181259 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 32414
Aug 10 06:37:16 dist-test-slave-4tdj krb5kdc[31994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.194@KRBTEST.COM
I20260810 06:37:16.197487 469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:16.182178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42384 (local address 127.31.59.194:44031)
0810 06:37:16.182957 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:16.182964 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:16.182990 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:37:16.183066 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:16.183072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:16.183274 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:16.183406 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:16.184576 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:16.185570 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:16.186922 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:16.187517 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:16.187625 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:16.192280 (+ 4655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:16.192292 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:16.192307 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:16.192345 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:16.194967 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:16.195574 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:16.195581 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:16.195583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:16.195705 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:16.196025 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:16.196028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:16.196030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:16.196364 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:16.196647 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:16.196931 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 06:37:16.197145 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":498,"threads_started":1}
I20260810 06:37:16.198930 32654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:16.279608 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 32414
I20260810 06:37:16.303012 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 32677
Aug 10 06:37:16 dist-test-slave-4tdj krb5kdc[31994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.195@KRBTEST.COM
I20260810 06:37:16.318868 475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:16.303872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48050 (local address 127.31.59.195:44633)
0810 06:37:16.304472 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:16.304480 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:16.304510 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:37:16.304582 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:16.304588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:16.304797 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:16.304928 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:16.306114 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:16.307274 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:16.308556 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:16.309144 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:16.309265 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:16.313360 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:16.313375 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:16.313389 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:16.313424 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:16.316269 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:16.316899 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:16.316907 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:16.316910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:16.317051 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:16.317356 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:16.317359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:16.317361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:16.317787 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:16.318040 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:16.318257 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 06:37:16.318562 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":284,"threads_started":1}
I20260810 06:37:16.320168 436 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:16.402983 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 32677
I20260810 06:37:16.423882 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 32031
I20260810 06:37:16.425028 32129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:16.525511 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 32031
2026-08-10T06:37:16Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4702 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:37:16 dist-test-slave-4tdj krb5kdc[480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:37:16 dist-test-slave-4tdj krb5kdc[480](info): set up 2 sockets
Aug 10 06:37:16 dist-test-slave-4tdj krb5kdc[480](info): commencing operation
krb5kdc: starting...
W20260810 06:37:17.724973 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.005s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:37:17 dist-test-slave-4tdj krb5kdc[480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:37:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:37:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:37:17.891943 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:35451
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:34551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:35451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:18.200172 504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:18.200457 504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:18.200600 504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:18.207532 504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:37:18.207639 504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:18.207717 504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:18.207790 504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:37:18.207852 504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:37:18.217231 504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:34551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:35451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:35451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:18.218523 504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:18.219786 504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:18.230880 513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:18.230988 512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:18.230899 516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:18.231926 504 server_base.cc:1034] running on GCE node
I20260810 06:37:18.232652 504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:18.233915 504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:18.235116 504 hybrid_clock.cc:648] HybridClock initialized: now 1786343838235066 us; error 40 us; skew 500 ppm
I20260810 06:37:18.235406 504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:18 dist-test-slave-4tdj krb5kdc[480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343838, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:18.238267 504 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:37:18.239696 504 webserver.cc:460] Webserver started at http://127.31.59.254:34493/ using document root <none> and password file <none>
I20260810 06:37:18.240314 504 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:18.240410 504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:18.240669 504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:18.243470 504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "a8fe7864b7124950adaa3078e4648989"
format_stamp: "Formatted at 2026-08-10 06:37:18 on dist-test-slave-4tdj"
server_key: "b5436125ac31e28071342f1327640fe7"
server_key_iv: "338ccfcb6bf2aa649492aaed5ea71d64"
server_key_version: "encryptionkey@0"
I20260810 06:37:18.244151 504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "a8fe7864b7124950adaa3078e4648989"
format_stamp: "Formatted at 2026-08-10 06:37:18 on dist-test-slave-4tdj"
server_key: "b5436125ac31e28071342f1327640fe7"
server_key_iv: "338ccfcb6bf2aa649492aaed5ea71d64"
server_key_version: "encryptionkey@0"
I20260810 06:37:18.249029 504 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:37:18.252266 528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:18.253481 504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:37:18.253641 504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "a8fe7864b7124950adaa3078e4648989"
format_stamp: "Formatted at 2026-08-10 06:37:18 on dist-test-slave-4tdj"
server_key: "b5436125ac31e28071342f1327640fe7"
server_key_iv: "338ccfcb6bf2aa649492aaed5ea71d64"
server_key_version: "encryptionkey@0"
I20260810 06:37:18.253806 504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:18.269147 504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:18.272934 504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:18.273191 504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:18.290071 504 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:35451
I20260810 06:37:18.290087 607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:35451 every 8 connection(s)
I20260810 06:37:18.291345 504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:37:18.294859 608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:18.295475 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 504
I20260810 06:37:18.295670 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:37:18.296262 31983 external_mini_cluster.cc:1428] Setting key 9f694b0f861bc8aa5b1e05390d4e25cd
I20260810 06:37:18.308269 608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989: Bootstrap starting.
Aug 10 06:37:18 dist-test-slave-4tdj krb5kdc[480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:18.311131 608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:18.312410 608 log.cc:824] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:18.315598 608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989: No bootstrap required, opened a new log
I20260810 06:37:18.319453 613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:18.298083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38344 (local address 127.31.59.254:35451)
0810 06:37:18.298886 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:18.298901 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:18.298951 (+ 50us) server_negotiation.cc:407] Connection header received
0810 06:37:18.300941 (+ 1990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:18.300969 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:18.301447 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:18.301864 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:18.303241 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:18.305159 (+ 1918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:18.306425 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:18.307434 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:18.307601 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:18.311241 (+ 3640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:18.311252 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:18.311270 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:18.311316 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:18.315029 (+ 3713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:18.315689 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:18.315697 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:18.315704 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:18.315907 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:18.316262 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:18.316268 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:18.316271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:18.316950 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:18.317265 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:18.317706 (+ 441us) server_negotiation.cc:299] Negotiation successful
0810 06:37:18.318140 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":326,"threads_started":1}
I20260810 06:37:18.324110 608 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8fe7864b7124950adaa3078e4648989" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35451 } }
I20260810 06:37:18.324450 608 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:18.324523 608 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8fe7864b7124950adaa3078e4648989, State: Initialized, Role: FOLLOWER
I20260810 06:37:18.324981 608 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8fe7864b7124950adaa3078e4648989" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35451 } }
I20260810 06:37:18.325181 608 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:37:18.325291 608 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:37:18.325425 608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:18.327759 608 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8fe7864b7124950adaa3078e4648989" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35451 } }
I20260810 06:37:18.328183 608 leader_election.cc:302] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8fe7864b7124950adaa3078e4648989; no voters:
I20260810 06:37:18.328751 608 leader_election.cc:288] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:37:18.328835 615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:18.329916 615 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989 [term 1 LEADER]: Becoming Leader. State: Replica: a8fe7864b7124950adaa3078e4648989, State: Running, Role: LEADER
I20260810 06:37:18.330446 615 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8fe7864b7124950adaa3078e4648989" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35451 } }
I20260810 06:37:18.330716 608 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:37:18.332993 624 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8fe7864b7124950adaa3078e4648989. Latest consensus state: current_term: 1 leader_uuid: "a8fe7864b7124950adaa3078e4648989" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8fe7864b7124950adaa3078e4648989" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35451 } } }
I20260810 06:37:18.333386 624 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:18.333230 621 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8fe7864b7124950adaa3078e4648989" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8fe7864b7124950adaa3078e4648989" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35451 } } }
I20260810 06:37:18.333540 621 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:18.335368 632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:37:18.340229 632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:37:18.348415 632 catalog_manager.cc:1347] Generated new cluster ID: f814942ebc05458ba4be6ddac4f6b346
I20260810 06:37:18.348521 632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:37:18.376577 632 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:37:18.377815 632 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:37:18.392859 632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8fe7864b7124950adaa3078e4648989: Generated new TSK 0
I20260810 06:37:18.393644 632 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
I20260810 06:37:18.446240 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:35451
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:34551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:18.761904 655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:18.762172 655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:18.762328 655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:18.769358 655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:18.769455 655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:18.769608 655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:37:18.778992 655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:34551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:35451
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:18.780208 655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:18.781507 655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:18.793972 667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:18.794071 662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:18.794025 663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:18.794842 655 server_base.cc:1034] running on GCE node
I20260810 06:37:18.795387 655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:18.796252 655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:18.797461 655 hybrid_clock.cc:648] HybridClock initialized: now 1786343838797399 us; error 68 us; skew 500 ppm
I20260810 06:37:18.797755 655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:18 dist-test-slave-4tdj krb5kdc[480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343838, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:18.800765 655 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:37:18.802089 655 webserver.cc:460] Webserver started at http://127.31.59.193:35195/ using document root <none> and password file <none>
I20260810 06:37:18.802772 655 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:18.802889 655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:18.803181 655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:18.805927 655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "0885ac02e4ae422594138f04a52f4053"
format_stamp: "Formatted at 2026-08-10 06:37:18 on dist-test-slave-4tdj"
server_key: "883a2cb08b008cb35376907b15aa7a1a"
server_key_iv: "0efc22a8cf6b43af9a93d5926a3a4681"
server_key_version: "encryptionkey@0"
I20260810 06:37:18.806701 655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "0885ac02e4ae422594138f04a52f4053"
format_stamp: "Formatted at 2026-08-10 06:37:18 on dist-test-slave-4tdj"
server_key: "883a2cb08b008cb35376907b15aa7a1a"
server_key_iv: "0efc22a8cf6b43af9a93d5926a3a4681"
server_key_version: "encryptionkey@0"
I20260810 06:37:18.811461 655 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 06:37:18.814752 680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:18.815979 655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:37:18.816123 655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "0885ac02e4ae422594138f04a52f4053"
format_stamp: "Formatted at 2026-08-10 06:37:18 on dist-test-slave-4tdj"
server_key: "883a2cb08b008cb35376907b15aa7a1a"
server_key_iv: "0efc22a8cf6b43af9a93d5926a3a4681"
server_key_version: "encryptionkey@0"
I20260810 06:37:18.816290 655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:18.832914 655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:18.836951 655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:18.837219 655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:18.838142 655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:18.839510 655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:18.839601 655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:18.839721 655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:18.839775 655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:18.868600 655 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:46631
I20260810 06:37:18.868635 854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:46631 every 8 connection(s)
I20260810 06:37:18.869848 655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:37:18.873440 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 655
I20260810 06:37:18.873641 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:37:18.874367 31983 external_mini_cluster.cc:1428] Setting key a210069aa12aa699795cba513f805030
Aug 10 06:37:18 dist-test-slave-4tdj krb5kdc[480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343838, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:18.889634 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:18.872390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:50041 (local address 127.31.59.254:35451)
0810 06:37:18.872881 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:18.872887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:18.873823 (+ 936us) server_negotiation.cc:407] Connection header received
0810 06:37:18.875885 (+ 2062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:18.875891 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:18.875974 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:18.876075 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:18.877547 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:18.878281 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:18.879807 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:18.880221 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:18.880330 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:18.884165 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:18.884170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:18.884173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:18.884205 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:18.886625 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:18.887484 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:18.887488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:18.887491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:18.887614 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:18.888063 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:18.888066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:18.888068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:18.888313 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:18.888418 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:18.889227 (+ 809us) server_negotiation.cc:299] Negotiation successful
0810 06:37:18.889384 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":236,"threads_started":1}
I20260810 06:37:18.891108 863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:18.872840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:35451 (local address 127.31.59.193:50041)
0810 06:37:18.873669 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:18.873716 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:18.875632 (+ 1916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:18.876312 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:18.876321 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:18.876829 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:18.877382 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:18.877399 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:18.878446 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:18.878450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:18.879669 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:18.879682 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:18.880434 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:18.880443 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:18.880691 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:18.881310 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:18.881338 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:18.883940 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:18.886749 (+ 2809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:18.886757 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:18.886771 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:18.887374 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:18.887715 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:18.887720 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:18.887723 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:18.887963 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:18.888432 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:18.888440 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:18.888875 (+ 435us) client_negotiation.cc:769] Sending connection context
0810 06:37:18.889193 (+ 318us) client_negotiation.cc:240] Negotiation successful
0810 06:37:18.889716 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":321,"threads_started":1}
WARNING: no policy specified for kudu/127.31.59.194@KRBTEST.COM; defaulting to no policy
I20260810 06:37:18.892768 856 heartbeater.cc:347] Connected to a master server at 127.31.59.254:35451
I20260810 06:37:18.893075 856 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:18.893780 856 heartbeater.cc:511] Master 127.31.59.254:35451 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.194@KRBTEST.COM" created.
I20260810 06:37:18.895912 560 ts_manager.cc:194] Registered new tserver with Master: 0885ac02e4ae422594138f04a52f4053 (127.31.59.193:46631)
I20260810 06:37:18.897338 560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:50041
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
I20260810 06:37:18.916747 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:35451
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:34551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:19.232185 871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:19.232441 871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:19.232578 871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:19.239637 871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:19.239737 871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:19.239876 871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:37:19.249145 871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:34551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:35451
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:19.250386 871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:19.251657 871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:19.263928 877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:19.264060 876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:19.263953 881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:19.264667 871 server_base.cc:1034] running on GCE node
I20260810 06:37:19.265156 871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:19.265956 871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:19.267140 871 hybrid_clock.cc:648] HybridClock initialized: now 1786343839267096 us; error 37 us; skew 500 ppm
I20260810 06:37:19.267390 871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:19 dist-test-slave-4tdj krb5kdc[480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343839, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:19.270309 871 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:37:19.271641 871 webserver.cc:460] Webserver started at http://127.31.59.194:42487/ using document root <none> and password file <none>
I20260810 06:37:19.272255 871 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:19.272364 871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:19.272655 871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:19.275475 871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "239005b2de7d4fdcaf669c52c2c15658"
format_stamp: "Formatted at 2026-08-10 06:37:19 on dist-test-slave-4tdj"
server_key: "16b2bf21855002975606e489f81d9c9d"
server_key_iv: "d777df3354c7b28693a97fb9706bd085"
server_key_version: "encryptionkey@0"
I20260810 06:37:19.276212 871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "239005b2de7d4fdcaf669c52c2c15658"
format_stamp: "Formatted at 2026-08-10 06:37:19 on dist-test-slave-4tdj"
server_key: "16b2bf21855002975606e489f81d9c9d"
server_key_iv: "d777df3354c7b28693a97fb9706bd085"
server_key_version: "encryptionkey@0"
I20260810 06:37:19.280957 871 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:37:19.284137 894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:19.285291 871 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260810 06:37:19.285440 871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "239005b2de7d4fdcaf669c52c2c15658"
format_stamp: "Formatted at 2026-08-10 06:37:19 on dist-test-slave-4tdj"
server_key: "16b2bf21855002975606e489f81d9c9d"
server_key_iv: "d777df3354c7b28693a97fb9706bd085"
server_key_version: "encryptionkey@0"
I20260810 06:37:19.285617 871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:19.313215 871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:19.317245 871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:19.317507 871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:19.318420 871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:19.319712 871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:19.319806 871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:19.319923 871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:19.319977 871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:19.348930 871 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:32885
I20260810 06:37:19.348974 1090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:32885 every 8 connection(s)
I20260810 06:37:19.350185 871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:37:19.352022 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 871
I20260810 06:37:19.352226 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:37:19.352651 31983 external_mini_cluster.cc:1428] Setting key 3c98950baf7a28bd7c2ccea3d237b6b7
Aug 10 06:37:19 dist-test-slave-4tdj krb5kdc[480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343839, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.59.195@KRBTEST.COM; defaulting to no policy
I20260810 06:37:19.370184 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:19.352895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:36475 (local address 127.31.59.254:35451)
0810 06:37:19.353106 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:19.353113 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:19.354363 (+ 1250us) server_negotiation.cc:407] Connection header received
0810 06:37:19.356176 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:19.356182 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:19.356273 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:19.356359 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:19.357896 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.358725 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.360221 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.360635 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.360744 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:19.364076 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:19.364083 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:19.364088 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:19.364127 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:19.367330 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:19.367984 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:19.367990 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:19.367994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:19.368120 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:19.368522 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:19.368527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:19.368530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:19.368776 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:19.368913 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:19.369608 (+ 695us) server_negotiation.cc:299] Negotiation successful
0810 06:37:19.369823 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:37:19.371085 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:19.353403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:35451 (local address 127.31.59.194:36475)
0810 06:37:19.354189 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:19.354235 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:19.355917 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:19.356631 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:19.356643 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:19.357130 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:19.357744 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:19.357764 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.358874 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.358880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:19.360085 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:19.360098 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.360848 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.360856 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:19.361075 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:19.361692 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:19.361713 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:19.363830 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:19.367455 (+ 3625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:19.367464 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:19.367479 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:19.367874 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:19.368229 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:19.368234 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:19.368238 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:19.368418 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:19.368897 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:19.368905 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:19.369230 (+ 325us) client_negotiation.cc:769] Sending connection context
0810 06:37:19.369466 (+ 236us) client_negotiation.cc:240] Negotiation successful
0810 06:37:19.369954 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":305,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.195@KRBTEST.COM" created.
I20260810 06:37:19.372510 1092 heartbeater.cc:347] Connected to a master server at 127.31.59.254:35451
I20260810 06:37:19.372843 1092 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:19.373670 1092 heartbeater.cc:511] Master 127.31.59.254:35451 requested a full tablet report, sending...
I20260810 06:37:19.375319 560 ts_manager.cc:194] Registered new tserver with Master: 239005b2de7d4fdcaf669c52c2c15658 (127.31.59.194:32885)
I20260810 06:37:19.376103 560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:36475
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
I20260810 06:37:19.393126 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:35451
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:34551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:19.704540 1103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:19.704793 1103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:19.704929 1103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:19.711932 1103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:19.712029 1103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:19.712181 1103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:37:19.721637 1103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:34551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:35451
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:19.722855 1103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:19.724159 1103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:19.736781 1112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:19.736925 1119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:19.737394 1117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:19.737469 1103 server_base.cc:1034] running on GCE node
I20260810 06:37:19.738087 1103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:19.739003 1103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:19.740213 1103 hybrid_clock.cc:648] HybridClock initialized: now 1786343839740159 us; error 46 us; skew 500 ppm
I20260810 06:37:19.740525 1103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:19 dist-test-slave-4tdj krb5kdc[480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343839, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:19.743475 1103 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:37:19.744769 1103 webserver.cc:460] Webserver started at http://127.31.59.195:39917/ using document root <none> and password file <none>
I20260810 06:37:19.745389 1103 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:19.745487 1103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:19.745752 1103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:19.748533 1103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "cf18a0b46c73407bba259bc0cb8a9a65"
format_stamp: "Formatted at 2026-08-10 06:37:19 on dist-test-slave-4tdj"
server_key: "4922fca418857c8c914049b930cb3c21"
server_key_iv: "5938a5825f9ab9bdc0b52cdbc1b6e305"
server_key_version: "encryptionkey@0"
I20260810 06:37:19.749229 1103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "cf18a0b46c73407bba259bc0cb8a9a65"
format_stamp: "Formatted at 2026-08-10 06:37:19 on dist-test-slave-4tdj"
server_key: "4922fca418857c8c914049b930cb3c21"
server_key_iv: "5938a5825f9ab9bdc0b52cdbc1b6e305"
server_key_version: "encryptionkey@0"
I20260810 06:37:19.753973 1103 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260810 06:37:19.757301 1134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:19.758608 1103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:37:19.758730 1103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "cf18a0b46c73407bba259bc0cb8a9a65"
format_stamp: "Formatted at 2026-08-10 06:37:19 on dist-test-slave-4tdj"
server_key: "4922fca418857c8c914049b930cb3c21"
server_key_iv: "5938a5825f9ab9bdc0b52cdbc1b6e305"
server_key_version: "encryptionkey@0"
I20260810 06:37:19.758890 1103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:19.775399 1103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:19.779047 1103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:19.779273 1103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:19.780107 1103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:19.781597 1103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:19.781667 1103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:19.781757 1103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:19.781800 1103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:19.811162 1103 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:39093
I20260810 06:37:19.811210 1339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:39093 every 8 connection(s)
I20260810 06:37:19.812381 1103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:37:19.818167 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 1103
I20260810 06:37:19.818357 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:37:19.818760 31983 external_mini_cluster.cc:1428] Setting key 6308d68e32af56a6bb6a63931ae1160b
Aug 10 06:37:19 dist-test-slave-4tdj krb5kdc[480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343839, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:19.830627 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:19.814783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:56143 (local address 127.31.59.254:35451)
0810 06:37:19.814980 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:19.814986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:19.816092 (+ 1106us) server_negotiation.cc:407] Connection header received
0810 06:37:19.817779 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:19.817783 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:19.817870 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:19.817963 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:19.819624 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.820561 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.822203 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.822651 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.822745 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:19.825759 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:19.825764 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:19.825767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:19.825798 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:19.828198 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:19.828880 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:19.828884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:19.828886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:19.828996 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:19.829311 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:19.829314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:19.829316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:19.829561 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:19.829667 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:19.830204 (+ 537us) server_negotiation.cc:299] Negotiation successful
0810 06:37:19.830398 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:37:19.831542 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:19.815266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:35451 (local address 127.31.59.195:56143)
0810 06:37:19.815944 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:19.815981 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:19.817534 (+ 1553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:19.818228 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:19.818239 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:19.818855 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:19.819435 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:19.819459 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.820710 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.820715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:19.822062 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:19.822075 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.822823 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.822828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:19.823022 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:19.823562 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:19.823582 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:19.825572 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:19.828326 (+ 2754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:19.828334 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:19.828349 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:19.828787 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:19.829070 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:19.829073 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:19.829076 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:19.829237 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:19.829647 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:19.829652 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:19.829926 (+ 274us) client_negotiation.cc:769] Sending connection context
0810 06:37:19.830149 (+ 223us) client_negotiation.cc:240] Negotiation successful
0810 06:37:19.830517 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":261,"threads_started":1}
I20260810 06:37:19.832988 1340 heartbeater.cc:347] Connected to a master server at 127.31.59.254:35451
I20260810 06:37:19.833278 1340 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:19.834028 1340 heartbeater.cc:511] Master 127.31.59.254:35451 requested a full tablet report, sending...
I20260810 06:37:19.835469 558 ts_manager.cc:194] Registered new tserver with Master: cf18a0b46c73407bba259bc0cb8a9a65 (127.31.59.195:39093)
I20260810 06:37:19.836300 558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:56143
I20260810 06:37:19.845803 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:37:19.866456 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:19.852817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38352 (local address 127.31.59.254:35451)
0810 06:37:19.852955 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:19.852959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:19.853593 (+ 634us) server_negotiation.cc:407] Connection header received
0810 06:37:19.853749 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:19.853754 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:19.853813 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:19.853890 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:19.855348 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.856203 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.857515 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.858259 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.858405 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:19.860958 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:19.860965 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:19.860970 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:19.861008 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:19.863925 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:19.864639 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:19.864643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:19.864645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:19.864786 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:19.865128 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:19.865131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:19.865133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:19.865469 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:19.865676 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:19.865995 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 06:37:19.866153 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:37:19.875367 558 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:37:19.883896 558 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:37:19.900383 856 heartbeater.cc:503] Master 127.31.59.254:35451 was elected leader, sending a full tablet report...
I20260810 06:37:19.924085 1367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:19.899589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53996 (local address 127.31.59.195:39093)
0810 06:37:19.912674 (+ 13085us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:19.912684 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:19.913417 (+ 733us) server_negotiation.cc:407] Connection header received
0810 06:37:19.913540 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:19.913547 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:19.913813 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:19.915844 (+ 2031us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:19.916124 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.917657 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.920986 (+ 3329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.922635 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.922722 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:19.922900 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:19.923516 (+ 616us) server_negotiation.cc:299] Negotiation successful
0810 06:37:19.923679 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12885,"thread_start_us":12661,"threads_started":1}
I20260810 06:37:19.924109 1361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:19.899309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:39093 (local address 127.0.0.1:53996)
0810 06:37:19.913129 (+ 13820us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:19.913156 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:19.913288 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:19.914071 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:19.914080 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:19.914112 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:19.915635 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:19.915649 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.917813 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.917820 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:19.920734 (+ 2914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:19.920751 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.923086 (+ 2335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.923091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:19.923248 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:19.923283 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 06:37:19.923400 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 06:37:19.923586 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13589,"mutex_wait_us":94,"thread_start_us":13344,"threads_started":1}
I20260810 06:37:19.924681 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:19.899039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:46631 (local address 127.0.0.1:34486)
0810 06:37:19.909162 (+ 10123us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:19.909219 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:19.909406 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:19.916875 (+ 7469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:19.916880 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:19.916905 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:19.917242 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:19.917252 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.919209 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.919215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:19.921980 (+ 2765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:19.921992 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.923890 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.923897 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:19.924078 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:19.924114 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:37:19.924178 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 06:37:19.924307 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9866,"mutex_wait_us":277,"thread_start_us":16189,"threads_started":1}
I20260810 06:37:19.925581 1363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:19.899435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34486 (local address 127.31.59.193:46631)
0810 06:37:19.916238 (+ 16803us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:19.916249 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:19.916285 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:37:19.916392 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:19.916402 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:19.916644 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:19.916866 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:19.917423 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.919083 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.922162 (+ 3079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.923662 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.924900 (+ 1238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:19.925116 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:19.925166 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 06:37:19.925323 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16598,"thread_start_us":17661,"threads_started":1}
I20260810 06:37:19.927404 1359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:19.902985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:32885 (local address 127.0.0.1:55336)
0810 06:37:19.914982 (+ 11997us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:19.915020 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:19.915154 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:19.918827 (+ 3673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:19.918832 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:19.918860 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:19.919974 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:19.919991 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.922117 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.922122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:19.924398 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:19.924412 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.926495 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.926504 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:19.926793 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:19.926831 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 06:37:19.926924 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 06:37:19.927052 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11842,"thread_start_us":5751,"threads_started":1}
I20260810 06:37:19.928733 1369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:19.903622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55336 (local address 127.31.59.194:32885)
0810 06:37:19.918264 (+ 14642us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:19.918272 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:19.918322 (+ 50us) server_negotiation.cc:407] Connection header received
0810 06:37:19.918408 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:19.918415 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:19.918626 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:19.918795 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:19.920210 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.921951 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.924573 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:19.926121 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:19.926263 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:19.926503 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:19.928253 (+ 1750us) server_negotiation.cc:299] Negotiation successful
0810 06:37:19.928420 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14475,"thread_start_us":14324,"threads_started":1}
I20260810 06:37:19.929721 1214 tablet_service.cc:1467] Processing CreateTablet for tablet 8c01b689156f4a9c8c366eff5e8518b8 (DEFAULT_TABLE table=test-table [id=2de3d04ff01e41dbbd020f2e654b47a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:19.929709 759 tablet_service.cc:1467] Processing CreateTablet for tablet 8c01b689156f4a9c8c366eff5e8518b8 (DEFAULT_TABLE table=test-table [id=2de3d04ff01e41dbbd020f2e654b47a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:19.931391 759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c01b689156f4a9c8c366eff5e8518b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:19.931437 1214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c01b689156f4a9c8c366eff5e8518b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:19.932489 985 tablet_service.cc:1467] Processing CreateTablet for tablet 8c01b689156f4a9c8c366eff5e8518b8 (DEFAULT_TABLE table=test-table [id=2de3d04ff01e41dbbd020f2e654b47a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:19.934453 985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c01b689156f4a9c8c366eff5e8518b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:19.944792 1371 tablet_bootstrap.cc:492] T 8c01b689156f4a9c8c366eff5e8518b8 P cf18a0b46c73407bba259bc0cb8a9a65: Bootstrap starting.
I20260810 06:37:19.949227 1371 tablet_bootstrap.cc:654] T 8c01b689156f4a9c8c366eff5e8518b8 P cf18a0b46c73407bba259bc0cb8a9a65: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:19.949434 1373 tablet_bootstrap.cc:492] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053: Bootstrap starting.
I20260810 06:37:19.950805 1371 log.cc:824] T 8c01b689156f4a9c8c366eff5e8518b8 P cf18a0b46c73407bba259bc0cb8a9a65: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:19.951063 1374 tablet_bootstrap.cc:492] T 8c01b689156f4a9c8c366eff5e8518b8 P 239005b2de7d4fdcaf669c52c2c15658: Bootstrap starting.
I20260810 06:37:19.953845 1373 tablet_bootstrap.cc:654] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:19.955183 1374 tablet_bootstrap.cc:654] T 8c01b689156f4a9c8c366eff5e8518b8 P 239005b2de7d4fdcaf669c52c2c15658: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:19.955415 1373 log.cc:824] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:19.956714 1371 tablet_bootstrap.cc:492] T 8c01b689156f4a9c8c366eff5e8518b8 P cf18a0b46c73407bba259bc0cb8a9a65: No bootstrap required, opened a new log
I20260810 06:37:19.956969 1371 ts_tablet_manager.cc:1397] T 8c01b689156f4a9c8c366eff5e8518b8 P cf18a0b46c73407bba259bc0cb8a9a65: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260810 06:37:19.957171 1374 log.cc:824] T 8c01b689156f4a9c8c366eff5e8518b8 P 239005b2de7d4fdcaf669c52c2c15658: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:19.958801 1373 tablet_bootstrap.cc:492] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053: No bootstrap required, opened a new log
I20260810 06:37:19.959126 1373 ts_tablet_manager.cc:1397] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260810 06:37:19.960057 1374 tablet_bootstrap.cc:492] T 8c01b689156f4a9c8c366eff5e8518b8 P 239005b2de7d4fdcaf669c52c2c15658: No bootstrap required, opened a new log
I20260810 06:37:19.960327 1374 ts_tablet_manager.cc:1397] T 8c01b689156f4a9c8c366eff5e8518b8 P 239005b2de7d4fdcaf669c52c2c15658: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 06:37:19.970199 1371 raft_consensus.cc:348] T 8c01b689156f4a9c8c366eff5e8518b8 P cf18a0b46c73407bba259bc0cb8a9a65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0885ac02e4ae422594138f04a52f4053" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46631 } } peers { permanent_uuid: "cf18a0b46c73407bba259bc0cb8a9a65" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 39093 } } peers { permanent_uuid: "239005b2de7d4fdcaf669c52c2c15658" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32885 } }
I20260810 06:37:19.970916 1371 raft_consensus.cc:374] T 8c01b689156f4a9c8c366eff5e8518b8 P cf18a0b46c73407bba259bc0cb8a9a65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:19.970695 1374 raft_consensus.cc:348] T 8c01b689156f4a9c8c366eff5e8518b8 P 239005b2de7d4fdcaf669c52c2c15658 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0885ac02e4ae422594138f04a52f4053" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46631 } } peers { permanent_uuid: "cf18a0b46c73407bba259bc0cb8a9a65" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 39093 } } peers { permanent_uuid: "239005b2de7d4fdcaf669c52c2c15658" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32885 } }
I20260810 06:37:19.971119 1371 raft_consensus.cc:729] T 8c01b689156f4a9c8c366eff5e8518b8 P cf18a0b46c73407bba259bc0cb8a9a65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf18a0b46c73407bba259bc0cb8a9a65, State: Initialized, Role: FOLLOWER
I20260810 06:37:19.971251 1374 raft_consensus.cc:374] T 8c01b689156f4a9c8c366eff5e8518b8 P 239005b2de7d4fdcaf669c52c2c15658 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:19.971387 1374 raft_consensus.cc:729] T 8c01b689156f4a9c8c366eff5e8518b8 P 239005b2de7d4fdcaf669c52c2c15658 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 239005b2de7d4fdcaf669c52c2c15658, State: Initialized, Role: FOLLOWER
I20260810 06:37:19.972018 1374 consensus_queue.cc:260] T 8c01b689156f4a9c8c366eff5e8518b8 P 239005b2de7d4fdcaf669c52c2c15658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0885ac02e4ae422594138f04a52f4053" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46631 } } peers { permanent_uuid: "cf18a0b46c73407bba259bc0cb8a9a65" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 39093 } } peers { permanent_uuid: "239005b2de7d4fdcaf669c52c2c15658" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32885 } }
I20260810 06:37:19.972102 1371 consensus_queue.cc:260] T 8c01b689156f4a9c8c366eff5e8518b8 P cf18a0b46c73407bba259bc0cb8a9a65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0885ac02e4ae422594138f04a52f4053" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46631 } } peers { permanent_uuid: "cf18a0b46c73407bba259bc0cb8a9a65" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 39093 } } peers { permanent_uuid: "239005b2de7d4fdcaf669c52c2c15658" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32885 } }
I20260810 06:37:19.972844 1373 raft_consensus.cc:348] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0885ac02e4ae422594138f04a52f4053" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46631 } } peers { permanent_uuid: "cf18a0b46c73407bba259bc0cb8a9a65" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 39093 } } peers { permanent_uuid: "239005b2de7d4fdcaf669c52c2c15658" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32885 } }
I20260810 06:37:19.973275 1092 heartbeater.cc:503] Master 127.31.59.254:35451 was elected leader, sending a full tablet report...
I20260810 06:37:19.973464 1373 raft_consensus.cc:374] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:19.973654 1373 raft_consensus.cc:729] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0885ac02e4ae422594138f04a52f4053, State: Initialized, Role: FOLLOWER
I20260810 06:37:19.973987 1340 heartbeater.cc:503] Master 127.31.59.254:35451 was elected leader, sending a full tablet report...
I20260810 06:37:19.974462 1373 consensus_queue.cc:260] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0885ac02e4ae422594138f04a52f4053" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46631 } } peers { permanent_uuid: "cf18a0b46c73407bba259bc0cb8a9a65" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 39093 } } peers { permanent_uuid: "239005b2de7d4fdcaf669c52c2c15658" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32885 } }
I20260810 06:37:19.975113 1371 ts_tablet_manager.cc:1428] T 8c01b689156f4a9c8c366eff5e8518b8 P cf18a0b46c73407bba259bc0cb8a9a65: Time spent starting tablet: real 0.018s user 0.014s sys 0.004s
I20260810 06:37:19.975701 1374 ts_tablet_manager.cc:1428] T 8c01b689156f4a9c8c366eff5e8518b8 P 239005b2de7d4fdcaf669c52c2c15658: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260810 06:37:19.978856 1373 ts_tablet_manager.cc:1428] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053: Time spent starting tablet: real 0.020s user 0.006s sys 0.012s
I20260810 06:37:20.034026 1383 raft_consensus.cc:482] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:37:20.034392 1383 raft_consensus.cc:504] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0885ac02e4ae422594138f04a52f4053" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46631 } } peers { permanent_uuid: "cf18a0b46c73407bba259bc0cb8a9a65" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 39093 } } peers { permanent_uuid: "239005b2de7d4fdcaf669c52c2c15658" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32885 } }
I20260810 06:37:20.037374 1383 leader_election.cc:288] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf18a0b46c73407bba259bc0cb8a9a65 (127.31.59.195:39093), 239005b2de7d4fdcaf669c52c2c15658 (127.31.59.194:32885)
I20260810 06:37:20.045280 1369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:20.038086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:40893 (local address 127.31.59.194:32885)
0810 06:37:20.038589 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:20.038595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:20.039609 (+ 1014us) server_negotiation.cc:407] Connection header received
0810 06:37:20.039795 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:20.039801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:20.039866 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:20.039941 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:20.040521 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.041553 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.043387 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.044176 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.044317 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:20.044395 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:20.044884 (+ 489us) server_negotiation.cc:299] Negotiation successful
0810 06:37:20.045006 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292}
I20260810 06:37:20.045768 1367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:20.038067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:45469 (local address 127.31.59.195:39093)
0810 06:37:20.038347 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:20.038353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:20.040327 (+ 1974us) server_negotiation.cc:407] Connection header received
0810 06:37:20.040514 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:20.040518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:20.040583 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:20.040658 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:20.041357 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.042107 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.043959 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.044786 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.044930 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:20.045001 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:20.045343 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 06:37:20.045466 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 06:37:20.045826 1384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:20.037519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:32885 (local address 127.31.59.193:40893)
0810 06:37:20.039499 (+ 1980us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:20.039520 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:20.039665 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:20.039981 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:20.039986 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:20.040014 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:20.040386 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:20.040396 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.041681 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.041687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:20.043233 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:20.043245 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.044531 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.044538 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:20.044724 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:20.044749 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 06:37:20.044835 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 06:37:20.044964 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1855,"mutex_wait_us":17,"thread_start_us":1562,"threads_started":1}
I20260810 06:37:20.046105 1387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:20.038316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:39093 (local address 127.31.59.193:45469)
0810 06:37:20.040213 (+ 1897us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:20.040231 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:20.040373 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:20.040683 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:20.040687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:20.040709 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:20.041176 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:20.041187 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.042242 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.042246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:20.043806 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:20.043819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.044997 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.045003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:20.045176 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:20.045201 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 06:37:20.045292 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 06:37:20.045410 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1771,"mutex_wait_us":20,"thread_start_us":1313,"threads_started":1}
I20260810 06:37:20.047171 1246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c01b689156f4a9c8c366eff5e8518b8" candidate_uuid: "0885ac02e4ae422594138f04a52f4053" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf18a0b46c73407bba259bc0cb8a9a65" is_pre_election: true
I20260810 06:37:20.047298 1019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c01b689156f4a9c8c366eff5e8518b8" candidate_uuid: "0885ac02e4ae422594138f04a52f4053" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239005b2de7d4fdcaf669c52c2c15658" is_pre_election: true
I20260810 06:37:20.048028 1019 raft_consensus.cc:2454] T 8c01b689156f4a9c8c366eff5e8518b8 P 239005b2de7d4fdcaf669c52c2c15658 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0885ac02e4ae422594138f04a52f4053 in term 0.
I20260810 06:37:20.048908 682 leader_election.cc:302] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0885ac02e4ae422594138f04a52f4053, 239005b2de7d4fdcaf669c52c2c15658; no voters:
I20260810 06:37:20.049329 1383 raft_consensus.cc:2781] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:37:20.049463 1383 raft_consensus.cc:482] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:37:20.049553 1383 raft_consensus.cc:3037] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:20.050264 1246 raft_consensus.cc:2454] T 8c01b689156f4a9c8c366eff5e8518b8 P cf18a0b46c73407bba259bc0cb8a9a65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0885ac02e4ae422594138f04a52f4053 in term 0.
I20260810 06:37:20.052191 1383 raft_consensus.cc:504] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0885ac02e4ae422594138f04a52f4053" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46631 } } peers { permanent_uuid: "cf18a0b46c73407bba259bc0cb8a9a65" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 39093 } } peers { permanent_uuid: "239005b2de7d4fdcaf669c52c2c15658" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32885 } }
I20260810 06:37:20.052876 1383 leader_election.cc:288] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053 [CANDIDATE]: Term 1 election: Requested vote from peers cf18a0b46c73407bba259bc0cb8a9a65 (127.31.59.195:39093), 239005b2de7d4fdcaf669c52c2c15658 (127.31.59.194:32885)
I20260810 06:37:20.053212 1246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c01b689156f4a9c8c366eff5e8518b8" candidate_uuid: "0885ac02e4ae422594138f04a52f4053" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf18a0b46c73407bba259bc0cb8a9a65"
I20260810 06:37:20.053308 1019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c01b689156f4a9c8c366eff5e8518b8" candidate_uuid: "0885ac02e4ae422594138f04a52f4053" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239005b2de7d4fdcaf669c52c2c15658"
I20260810 06:37:20.053421 1246 raft_consensus.cc:3037] T 8c01b689156f4a9c8c366eff5e8518b8 P cf18a0b46c73407bba259bc0cb8a9a65 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:20.053495 1019 raft_consensus.cc:3037] T 8c01b689156f4a9c8c366eff5e8518b8 P 239005b2de7d4fdcaf669c52c2c15658 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:20.056932 1246 raft_consensus.cc:2454] T 8c01b689156f4a9c8c366eff5e8518b8 P cf18a0b46c73407bba259bc0cb8a9a65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0885ac02e4ae422594138f04a52f4053 in term 1.
I20260810 06:37:20.056952 1019 raft_consensus.cc:2454] T 8c01b689156f4a9c8c366eff5e8518b8 P 239005b2de7d4fdcaf669c52c2c15658 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0885ac02e4ae422594138f04a52f4053 in term 1.
I20260810 06:37:20.057369 688 leader_election.cc:302] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0885ac02e4ae422594138f04a52f4053, cf18a0b46c73407bba259bc0cb8a9a65; no voters:
I20260810 06:37:20.057715 1383 raft_consensus.cc:2781] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:20.058260 1383 raft_consensus.cc:686] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053 [term 1 LEADER]: Becoming Leader. State: Replica: 0885ac02e4ae422594138f04a52f4053, State: Running, Role: LEADER
I20260810 06:37:20.058743 1383 consensus_queue.cc:237] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0885ac02e4ae422594138f04a52f4053" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46631 } } peers { permanent_uuid: "cf18a0b46c73407bba259bc0cb8a9a65" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 39093 } } peers { permanent_uuid: "239005b2de7d4fdcaf669c52c2c15658" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32885 } }
W20260810 06:37:20.063817 1341 tablet.cc:2369] T 8c01b689156f4a9c8c366eff5e8518b8 P cf18a0b46c73407bba259bc0cb8a9a65: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:37:20.064543 558 catalog_manager.cc:5495] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0885ac02e4ae422594138f04a52f4053 (127.31.59.193). New cstate: current_term: 1 leader_uuid: "0885ac02e4ae422594138f04a52f4053" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0885ac02e4ae422594138f04a52f4053" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf18a0b46c73407bba259bc0cb8a9a65" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 39093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "239005b2de7d4fdcaf669c52c2c15658" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32885 } health_report { overall_health: UNKNOWN } } }
Aug 10 06:37:20 dist-test-slave-4tdj krb5kdc[480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
W20260810 06:37:20.102129 1093 tablet.cc:2369] T 8c01b689156f4a9c8c366eff5e8518b8 P 239005b2de7d4fdcaf669c52c2c15658: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:37:20.109272 1363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:20.091996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34500 (local address 127.31.59.193:46631)
0810 06:37:20.092288 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:20.092295 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:20.093274 (+ 979us) server_negotiation.cc:407] Connection header received
0810 06:37:20.093456 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:20.093460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:20.093524 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:20.093658 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:20.095779 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.096782 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.098179 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.098620 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.098707 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:20.102832 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:20.102847 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:20.102863 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:20.102900 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:20.106026 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:20.106803 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:20.106811 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:20.106814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:20.107105 (+ 291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:20.107521 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:20.107525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:20.107527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:20.107891 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:20.108214 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:20.108599 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 06:37:20.108878 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260810 06:37:20.122236 857 tablet.cc:2369] T 8c01b689156f4a9c8c366eff5e8518b8 P 0885ac02e4ae422594138f04a52f4053: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 06:37:20 dist-test-slave-4tdj krb5kdc[480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343840, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:37:20 dist-test-slave-4tdj krb5kdc[480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343840, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:37:20.158089 1363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:20.143727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34504 (local address 127.31.59.193:46631)
0810 06:37:20.143948 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:20.143954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:20.144126 (+ 172us) server_negotiation.cc:407] Connection header received
0810 06:37:20.144310 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:20.144314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:20.144379 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:20.144464 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:20.145980 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.146789 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.148225 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.148667 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.148779 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:20.153097 (+ 4318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:20.153103 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:20.153107 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:20.153141 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:20.155412 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:20.155985 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:20.155991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:20.156008 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:20.156129 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:20.156498 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:20.156502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:20.156505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:20.156755 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:20.157142 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:20.157278 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 06:37:20.157533 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Aug 10 06:37:20 dist-test-slave-4tdj krb5kdc[480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:37:20 dist-test-slave-4tdj krb5kdc[480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:37:20.206686 1363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:20.191131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34506 (local address 127.31.59.193:46631)
0810 06:37:20.191324 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:20.191330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:20.191736 (+ 406us) server_negotiation.cc:407] Connection header received
0810 06:37:20.191919 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:20.191923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:20.192062 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:20.192156 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:20.193559 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.194503 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.195901 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.196584 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.196749 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:20.201096 (+ 4347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:20.201101 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:20.201105 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:20.201139 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:20.203842 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:20.204576 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:20.204581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:20.204583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:20.204707 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:20.205099 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:20.205104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:20.205107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:20.205389 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:20.205642 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:20.205941 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 06:37:20.206171 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260810 06:37:20.208885 724 tablet_service.cc:2908] Rejecting scan request for tablet 8c01b689156f4a9c8c366eff5e8518b8: Uninitialized: clean time has not yet been initialized
I20260810 06:37:20.212610 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 655
I20260810 06:37:20.225045 1363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:20.213564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34510 (local address 127.31.59.193:46631)
0810 06:37:20.213709 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:20.213712 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:20.214033 (+ 321us) server_negotiation.cc:407] Connection header received
0810 06:37:20.214164 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:20.214167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:20.214238 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:20.214348 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:20.215422 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.216124 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.217209 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.217583 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.217671 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:20.219939 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:20.219945 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:20.219949 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:20.219989 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:20.222885 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:20.223472 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:20.223476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:20.223479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:20.223608 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:20.223884 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:20.223888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:20.223889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:20.224168 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:20.224391 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:20.224554 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 06:37:20.224756 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:37:20.225765 849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:20.316133 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 655
I20260810 06:37:20.342851 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 871
Aug 10 06:37:20 dist-test-slave-4tdj krb5kdc[480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.194@KRBTEST.COM
I20260810 06:37:20.357870 1369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:20.343623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55346 (local address 127.31.59.194:32885)
0810 06:37:20.343844 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:20.343851 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:20.343938 (+ 87us) server_negotiation.cc:407] Connection header received
0810 06:37:20.344142 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:20.344146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:20.344213 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:20.344301 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:20.345728 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.346523 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.347809 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.348211 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.348321 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:20.352204 (+ 3883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:20.352215 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:20.352228 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:20.352267 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:20.355222 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:20.355837 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:20.355844 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:20.355846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:20.355975 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:20.356340 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:20.356344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:20.356346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:20.356739 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:20.357025 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:20.357311 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 06:37:20.357571 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 06:37:20.358556 1082 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:20.445372 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 871
I20260810 06:37:20.467723 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 1103
Aug 10 06:37:20 dist-test-slave-4tdj krb5kdc[480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.195@KRBTEST.COM
I20260810 06:37:20.481904 1367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:20.468510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54008 (local address 127.31.59.195:39093)
0810 06:37:20.468735 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:20.468741 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:20.468762 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:37:20.468830 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:20.468834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:20.468897 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:20.468982 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:20.470183 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.470993 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.472270 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:20.472672 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:20.472779 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:20.476789 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:20.476802 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:20.476818 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:20.476858 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:20.479358 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:20.479964 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:20.479972 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:20.479975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:20.480105 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:20.480452 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:20.480455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:20.480457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:20.480803 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:20.481082 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:20.481358 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 06:37:20.481594 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260810 06:37:20.482553 1332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:20.566867 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 1103
I20260810 06:37:20.586464 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 504
I20260810 06:37:20.587626 601 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:20.695746 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 504
2026-08-10T06:37:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4171 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:37:20 dist-test-slave-4tdj krb5kdc[1420](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:37:20 dist-test-slave-4tdj krb5kdc[1420](info): set up 2 sockets
Aug 10 06:37:20 dist-test-slave-4tdj krb5kdc[1420](info): commencing operation
krb5kdc: starting...
W20260810 06:37:21.905573 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:37:22 dist-test-slave-4tdj krb5kdc[1420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:37:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:37:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:37:22.083161 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:35305
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:35415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:35305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:22.397589 1442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:22.397838 1442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:22.397933 1442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:22.404762 1442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:37:22.404855 1442 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 06:37:22.404907 1442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:22.404966 1442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:22.405014 1442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:37:22.405073 1442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:37:22.414268 1442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:35415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:35305
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:35305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:22.415340 1442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:22.416534 1442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:22.427747 1457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:22.427754 1456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:22.428195 1461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:22.428639 1442 server_base.cc:1034] running on GCE node
I20260810 06:37:22.429365 1442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:22.430706 1442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:22.431962 1442 hybrid_clock.cc:648] HybridClock initialized: now 1786343842431896 us; error 28 us; skew 500 ppm
I20260810 06:37:22.432314 1442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:22 dist-test-slave-4tdj krb5kdc[1420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343842, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:22.435581 1442 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:37:22.437094 1442 webserver.cc:460] Webserver started at http://127.31.59.254:37455/ using document root <none> and password file <none>
I20260810 06:37:22.437803 1442 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:22.437928 1442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:22.438277 1442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:22.441322 1442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "ccb6530327534bb791e1e5b7c4c5efd5"
format_stamp: "Formatted at 2026-08-10 06:37:22 on dist-test-slave-4tdj"
server_key: "779ec3bddd33401729fd7620ddd8fd5f"
server_key_iv: "e3a9f11b142f5e60be9dad8cfd20964b"
server_key_version: "encryptionkey@0"
I20260810 06:37:22.442135 1442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "ccb6530327534bb791e1e5b7c4c5efd5"
format_stamp: "Formatted at 2026-08-10 06:37:22 on dist-test-slave-4tdj"
server_key: "779ec3bddd33401729fd7620ddd8fd5f"
server_key_iv: "e3a9f11b142f5e60be9dad8cfd20964b"
server_key_version: "encryptionkey@0"
I20260810 06:37:22.447506 1442 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 06:37:22.451005 1472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:22.452320 1442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260810 06:37:22.452495 1442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "ccb6530327534bb791e1e5b7c4c5efd5"
format_stamp: "Formatted at 2026-08-10 06:37:22 on dist-test-slave-4tdj"
server_key: "779ec3bddd33401729fd7620ddd8fd5f"
server_key_iv: "e3a9f11b142f5e60be9dad8cfd20964b"
server_key_version: "encryptionkey@0"
I20260810 06:37:22.452688 1442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:22.479219 1442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:22.483141 1442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:22.483381 1442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:22.504016 1442 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:35305
I20260810 06:37:22.504034 1578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:35305 every 8 connection(s)
I20260810 06:37:22.505316 1442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:37:22.508477 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 1442
I20260810 06:37:22.508674 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:37:22.509018 1580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:22.509356 31983 external_mini_cluster.cc:1428] Setting key 5db4e997f7196a3d03d75c0af7f2d775
I20260810 06:37:22.523366 1580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5: Bootstrap starting.
Aug 10 06:37:22 dist-test-slave-4tdj krb5kdc[1420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:22.527029 1580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:22.528143 1580 log.cc:824] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:22.530972 1580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5: No bootstrap required, opened a new log
I20260810 06:37:22.531893 1585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:22.511198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51392 (local address 127.31.59.254:35305)
0810 06:37:22.512049 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:22.512065 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:22.512119 (+ 54us) server_negotiation.cc:407] Connection header received
0810 06:37:22.513976 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:22.514006 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:22.514541 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:22.514969 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:22.516202 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:22.518162 (+ 1960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:22.519423 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:22.520583 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:22.520814 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:22.524865 (+ 4051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:22.524876 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:22.524893 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:22.524939 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:22.527851 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:22.528609 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:22.528616 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:22.528621 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:22.528788 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:22.529176 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:22.529180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:22.529185 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:22.529692 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:22.529955 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:22.530453 (+ 498us) server_negotiation.cc:299] Negotiation successful
0810 06:37:22.530867 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":367,"threads_started":1}
I20260810 06:37:22.538216 1580 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccb6530327534bb791e1e5b7c4c5efd5" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35305 } }
I20260810 06:37:22.538570 1580 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:22.538664 1580 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb6530327534bb791e1e5b7c4c5efd5, State: Initialized, Role: FOLLOWER
I20260810 06:37:22.539215 1580 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccb6530327534bb791e1e5b7c4c5efd5" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35305 } }
I20260810 06:37:22.539481 1580 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:37:22.539639 1580 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:37:22.539788 1580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:22.541785 1580 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccb6530327534bb791e1e5b7c4c5efd5" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35305 } }
I20260810 06:37:22.542241 1580 leader_election.cc:302] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccb6530327534bb791e1e5b7c4c5efd5; no voters:
I20260810 06:37:22.542798 1580 leader_election.cc:288] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:37:22.542932 1588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:22.543634 1588 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5 [term 1 LEADER]: Becoming Leader. State: Replica: ccb6530327534bb791e1e5b7c4c5efd5, State: Running, Role: LEADER
I20260810 06:37:22.544171 1588 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccb6530327534bb791e1e5b7c4c5efd5" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35305 } }
I20260810 06:37:22.544708 1580 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:37:22.546900 1589 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccb6530327534bb791e1e5b7c4c5efd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccb6530327534bb791e1e5b7c4c5efd5" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35305 } } }
I20260810 06:37:22.546886 1591 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccb6530327534bb791e1e5b7c4c5efd5. Latest consensus state: current_term: 1 leader_uuid: "ccb6530327534bb791e1e5b7c4c5efd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccb6530327534bb791e1e5b7c4c5efd5" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35305 } } }
I20260810 06:37:22.547323 1591 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:22.547406 1589 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:22.553009 1596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:37:22.558096 1596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:37:22.575326 1596 catalog_manager.cc:1347] Generated new cluster ID: f6dc7c6ad9ab4e2f89e80bb5ce19c599
I20260810 06:37:22.575464 1596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:37:22.605726 1596 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:37:22.607167 1596 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:37:22.621353 1596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5: Generated new TSK 0
I20260810 06:37:22.622531 1596 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
I20260810 06:37:22.665755 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:35305
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:35415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:22.993633 1627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:22.993928 1627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:22.994077 1627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:23.001390 1627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:23.001525 1627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:23.001672 1627 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:37:23.001801 1627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:37:23.011592 1627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:35415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.59.254:35305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:23.012882 1627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:23.014240 1627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:23.029309 1637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:23.029382 1638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:23.029327 1643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:23.030331 1627 server_base.cc:1034] running on GCE node
I20260810 06:37:23.030965 1627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:23.031984 1627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:23.033212 1627 hybrid_clock.cc:648] HybridClock initialized: now 1786343843033124 us; error 46 us; skew 500 ppm
I20260810 06:37:23.033488 1627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:23 dist-test-slave-4tdj krb5kdc[1420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:23.036444 1627 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:37:23.037997 1627 webserver.cc:460] Webserver started at http://127.31.59.193:43913/ using document root <none> and password file <none>
I20260810 06:37:23.038748 1627 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:23.038848 1627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:23.039152 1627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:23.042097 1627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "d740a1d4acb946028e53c02e191b30cd"
format_stamp: "Formatted at 2026-08-10 06:37:23 on dist-test-slave-4tdj"
server_key: "e87140854b51025f9c160e6b8c4ae957"
server_key_iv: "692451eab7d3e33919580ad0b4743fec"
server_key_version: "encryptionkey@0"
I20260810 06:37:23.042999 1627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "d740a1d4acb946028e53c02e191b30cd"
format_stamp: "Formatted at 2026-08-10 06:37:23 on dist-test-slave-4tdj"
server_key: "e87140854b51025f9c160e6b8c4ae957"
server_key_iv: "692451eab7d3e33919580ad0b4743fec"
server_key_version: "encryptionkey@0"
I20260810 06:37:23.048508 1627 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 06:37:23.051982 1652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:23.053275 1627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:37:23.053439 1627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "d740a1d4acb946028e53c02e191b30cd"
format_stamp: "Formatted at 2026-08-10 06:37:23 on dist-test-slave-4tdj"
server_key: "e87140854b51025f9c160e6b8c4ae957"
server_key_iv: "692451eab7d3e33919580ad0b4743fec"
server_key_version: "encryptionkey@0"
I20260810 06:37:23.053627 1627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:23.071951 1627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:23.076066 1627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:23.076308 1627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:23.079177 1627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:23.079280 1627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:23.079420 1627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:23.079488 1627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:37:23 dist-test-slave-4tdj krb5kdc[1420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:23.103158 1676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.080836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:58353 (local address 127.31.59.254:35305)
0810 06:37:23.081872 (+ 1036us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:23.081878 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:23.081901 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:37:23.083760 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:23.083768 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:23.083885 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:23.084002 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:23.086057 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.087017 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.089405 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.090070 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.090207 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.095027 (+ 4820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:23.095034 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:23.095039 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:23.095083 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:23.098191 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:23.099121 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:23.099127 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:23.099131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:23.099267 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:23.099827 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:23.099832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:23.099836 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:23.100211 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:23.100361 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:23.102491 (+ 2130us) server_negotiation.cc:299] Negotiation successful
0810 06:37:23.102771 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":919,"thread_start_us":816,"threads_started":1}
I20260810 06:37:23.103159 1674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.080284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:35305 (local address 127.31.59.193:58353)
0810 06:37:23.081131 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:23.081176 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:23.083391 (+ 2215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:23.084295 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:23.084302 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:23.085018 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:23.085882 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.085906 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.087222 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.087225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.089226 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.089241 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.090353 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.090363 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.090657 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.091316 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:23.091338 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:23.094756 (+ 3418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:23.098343 (+ 3587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:23.098351 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:23.098366 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:23.098994 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:23.099381 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:23.099389 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:23.099393 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:23.099712 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:23.100427 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:23.100432 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:23.100906 (+ 474us) client_negotiation.cc:769] Sending connection context
0810 06:37:23.101185 (+ 279us) client_negotiation.cc:240] Negotiation successful
0810 06:37:23.101731 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":336,"threads_started":1}
I20260810 06:37:23.120002 1627 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:40139
I20260810 06:37:23.120021 1846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:40139 every 8 connection(s)
I20260810 06:37:23.121335 1627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:37:23.123204 1810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.109833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:35305 (local address 127.31.59.193:55621)
0810 06:37:23.110929 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:23.110945 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:23.111062 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:23.111383 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:23.111387 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:23.111810 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:23.112255 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.112273 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.113563 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.113572 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.115033 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.115049 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.115856 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.115866 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.116048 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.116677 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:23.116694 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:23.117692 (+ 998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:23.120646 (+ 2954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:23.120652 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:23.120656 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:23.121041 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:23.121409 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:23.121414 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:23.121417 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:23.121564 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:23.122055 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:23.122059 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:23.122244 (+ 185us) client_negotiation.cc:769] Sending connection context
0810 06:37:23.122423 (+ 179us) client_negotiation.cc:240] Negotiation successful
0810 06:37:23.122770 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"thread_start_us":840,"threads_started":1}
I20260810 06:37:23.123440 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 1627
I20260810 06:37:23.123657 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:37:23.123634 1849 heartbeater.cc:347] Connected to a master server at 127.31.59.254:35305
I20260810 06:37:23.123872 1849 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:23.124117 1676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.110201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:55621 (local address 127.31.59.254:35305)
0810 06:37:23.110399 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:23.110405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:23.111014 (+ 609us) server_negotiation.cc:407] Connection header received
0810 06:37:23.111175 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:23.111180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:23.111261 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:23.111363 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:23.112455 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.113432 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.115174 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.115670 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.115792 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.117860 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:23.117866 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:23.117871 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:23.117902 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:23.120528 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:23.121147 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:23.121152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:23.121156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:23.121283 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:23.121668 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:23.121672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:23.121674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:23.121926 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:23.122061 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:23.123527 (+ 1466us) server_negotiation.cc:299] Negotiation successful
0810 06:37:23.123761 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:37:23.124408 31983 external_mini_cluster.cc:1428] Setting key c25b6aaf617b2875b63c2441a660c37d
I20260810 06:37:23.124680 1849 heartbeater.cc:511] Master 127.31.59.254:35305 requested a full tablet report, sending...
I20260810 06:37:23.127190 1500 ts_manager.cc:194] Registered new tserver with Master: d740a1d4acb946028e53c02e191b30cd (127.31.59.193:40139)
I20260810 06:37:23.128767 1500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:58353
WARNING: no policy specified for kudu/127.31.59.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
I20260810 06:37:23.163147 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:35305
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:35415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:23.474673 1856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:23.474944 1856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:23.475068 1856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:23.482188 1856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:23.482321 1856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:23.482438 1856 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:37:23.482556 1856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:37:23.492270 1856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:35415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.59.254:35305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:23.493547 1856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:23.494894 1856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:23.507652 1869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:23.507745 1868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:23.508369 1871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:23.508831 1856 server_base.cc:1034] running on GCE node
I20260810 06:37:23.509424 1856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:23.510344 1856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:23.511577 1856 hybrid_clock.cc:648] HybridClock initialized: now 1786343843511498 us; error 54 us; skew 500 ppm
I20260810 06:37:23.511863 1856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:23 dist-test-slave-4tdj krb5kdc[1420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:23.515228 1856 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:37:23.517097 1856 webserver.cc:460] Webserver started at http://127.31.59.194:33269/ using document root <none> and password file <none>
I20260810 06:37:23.517976 1856 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:23.518083 1856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:23.518462 1856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:23.522173 1856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "9d53289a6c73403ba1a5b37cf926fcb8"
format_stamp: "Formatted at 2026-08-10 06:37:23 on dist-test-slave-4tdj"
server_key: "a6e4f2441cb1a4cefeacf4f2e8b60d62"
server_key_iv: "b5045098b75eb60e6aef60cc49153908"
server_key_version: "encryptionkey@0"
I20260810 06:37:23.522972 1856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "9d53289a6c73403ba1a5b37cf926fcb8"
format_stamp: "Formatted at 2026-08-10 06:37:23 on dist-test-slave-4tdj"
server_key: "a6e4f2441cb1a4cefeacf4f2e8b60d62"
server_key_iv: "b5045098b75eb60e6aef60cc49153908"
server_key_version: "encryptionkey@0"
I20260810 06:37:23.527998 1856 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:37:23.531327 1882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:23.532713 1856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:37:23.532858 1856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "9d53289a6c73403ba1a5b37cf926fcb8"
format_stamp: "Formatted at 2026-08-10 06:37:23 on dist-test-slave-4tdj"
server_key: "a6e4f2441cb1a4cefeacf4f2e8b60d62"
server_key_iv: "b5045098b75eb60e6aef60cc49153908"
server_key_version: "encryptionkey@0"
I20260810 06:37:23.533036 1856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:23.556339 1856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:23.560787 1856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:23.561051 1856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:23.563670 1856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:23.563762 1856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:23.563889 1856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:23.563948 1856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:37:23 dist-test-slave-4tdj krb5kdc[1420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:23.586364 1676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.564415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:54153 (local address 127.31.59.254:35305)
0810 06:37:23.564616 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:23.564622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:23.566075 (+ 1453us) server_negotiation.cc:407] Connection header received
0810 06:37:23.568328 (+ 2253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:23.568334 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:23.568423 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:23.568512 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:23.570682 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.571643 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.573905 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.574496 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.574631 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.579387 (+ 4756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:23.579392 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:23.579395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:23.579431 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:23.582526 (+ 3095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:23.583442 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:23.583446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:23.583449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:23.583568 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:23.584171 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:23.584177 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:23.584180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:23.584473 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:23.584601 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:23.585724 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0810 06:37:23.585975 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:37:23.588243 1900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.564992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:35305 (local address 127.31.59.194:54153)
0810 06:37:23.565869 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:23.565928 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:23.567988 (+ 2060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:23.568797 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:23.568812 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:23.569549 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:23.570421 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.570452 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.571813 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.571819 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.573756 (+ 1937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.573769 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.574767 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.574775 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.575111 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.575922 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:23.575959 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:23.579176 (+ 3217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:23.582678 (+ 3502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:23.582691 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:23.582715 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:23.583264 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:23.583679 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:23.583685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:23.583690 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:23.584026 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:23.584606 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:23.584615 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:23.585180 (+ 565us) client_negotiation.cc:769] Sending connection context
0810 06:37:23.585563 (+ 383us) client_negotiation.cc:240] Negotiation successful
0810 06:37:23.586203 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":363,"threads_started":1}
I20260810 06:37:23.605273 1856 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:32979
I20260810 06:37:23.605408 2093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:32979 every 8 connection(s)
I20260810 06:37:23.606581 1856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:37:23.608614 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 1856
I20260810 06:37:23.608724 2094 heartbeater.cc:347] Connected to a master server at 127.31.59.254:35305
I20260810 06:37:23.608812 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:37:23.608958 2094 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:23.609268 31983 external_mini_cluster.cc:1428] Setting key 8cced86e369b8ee4d486ded8c29c2748
I20260810 06:37:23.609797 2094 heartbeater.cc:511] Master 127.31.59.254:35305 requested a full tablet report, sending...
I20260810 06:37:23.611349 1500 ts_manager.cc:194] Registered new tserver with Master: 9d53289a6c73403ba1a5b37cf926fcb8 (127.31.59.194:32979)
I20260810 06:37:23.612115 1500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:54153
I20260810 06:37:23.612171 2063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.594953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:35305 (local address 127.31.59.194:49869)
0810 06:37:23.598818 (+ 3865us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:23.598841 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:23.599000 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:23.599328 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:23.599340 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:23.599999 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:23.600479 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.600490 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.601691 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.601698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.603463 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.603479 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.604192 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.604202 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.604377 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.605022 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:23.605040 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:23.605993 (+ 953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:23.609410 (+ 3417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:23.609415 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:23.609419 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:23.609928 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:23.610389 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:23.610393 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:23.610396 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:23.610558 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:23.611172 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:23.611177 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:23.611335 (+ 158us) client_negotiation.cc:769] Sending connection context
0810 06:37:23.611477 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 06:37:23.611753 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3699,"thread_start_us":3520,"threads_started":1}
I20260810 06:37:23.612933 1676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.595043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:49869 (local address 127.31.59.254:35305)
0810 06:37:23.595252 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:23.595259 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:23.598933 (+ 3674us) server_negotiation.cc:407] Connection header received
0810 06:37:23.599127 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:23.599131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:23.599190 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:23.599272 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:23.600618 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.601519 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.603597 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.603989 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.604112 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.606171 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:23.606175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:23.606178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:23.606208 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:23.609259 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:23.610090 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:23.610096 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:23.610099 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:23.610261 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:23.610669 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:23.610674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:23.610677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:23.611043 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:23.611182 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:23.612434 (+ 1252us) server_negotiation.cc:299] Negotiation successful
0810 06:37:23.612626 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
WARNING: no policy specified for kudu/127.31.59.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
I20260810 06:37:23.648880 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:35305
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:35415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:23.964077 2104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:23.964352 2104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:23.964471 2104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:23.971676 2104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:23.971791 2104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:23.971891 2104 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:37:23.971977 2104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:37:23.981782 2104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:35415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.59.254:35305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:23.983050 2104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:23.984433 2104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:23.997207 2109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:23.997179 2110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:23.997362 2114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:23.998198 2104 server_base.cc:1034] running on GCE node
I20260810 06:37:23.998805 2104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:23.999640 2104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:24.000870 2104 hybrid_clock.cc:648] HybridClock initialized: now 1786343844000807 us; error 35 us; skew 500 ppm
I20260810 06:37:24.001173 2104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:23 dist-test-slave-4tdj krb5kdc[1420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:24.004261 2104 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:37:24.005761 2104 webserver.cc:460] Webserver started at http://127.31.59.195:33111/ using document root <none> and password file <none>
I20260810 06:37:24.006465 2104 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:24.006564 2104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:24.006850 2104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:24.009765 2104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "54edb32f730f45e7974a7a9068a7a906"
format_stamp: "Formatted at 2026-08-10 06:37:24 on dist-test-slave-4tdj"
server_key: "6c8608721a34a2d46e5c6b0b49155ade"
server_key_iv: "9c35aca5eb91138fe2f301995d0113e2"
server_key_version: "encryptionkey@0"
I20260810 06:37:24.010723 2104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "54edb32f730f45e7974a7a9068a7a906"
format_stamp: "Formatted at 2026-08-10 06:37:24 on dist-test-slave-4tdj"
server_key: "6c8608721a34a2d46e5c6b0b49155ade"
server_key_iv: "9c35aca5eb91138fe2f301995d0113e2"
server_key_version: "encryptionkey@0"
I20260810 06:37:24.015707 2104 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 06:37:24.019142 2121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:24.020474 2104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:37:24.020638 2104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "54edb32f730f45e7974a7a9068a7a906"
format_stamp: "Formatted at 2026-08-10 06:37:24 on dist-test-slave-4tdj"
server_key: "6c8608721a34a2d46e5c6b0b49155ade"
server_key_iv: "9c35aca5eb91138fe2f301995d0113e2"
server_key_version: "encryptionkey@0"
I20260810 06:37:24.020859 2104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:24.038654 2104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:24.042428 2104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:24.042665 2104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:24.045305 2104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:24.045404 2104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:24.045535 2104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:24.045605 2104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:37:24 dist-test-slave-4tdj krb5kdc[1420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:24.066700 1676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:24.045868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:57659 (local address 127.31.59.254:35305)
0810 06:37:24.046062 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:24.046069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:24.047512 (+ 1443us) server_negotiation.cc:407] Connection header received
0810 06:37:24.049601 (+ 2089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:24.049607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:24.049693 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:24.049783 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:24.051912 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.053048 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.055501 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.056219 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.056359 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:24.060309 (+ 3950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:24.060314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:24.060317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:24.060353 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:24.063147 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:24.064095 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:24.064102 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:24.064106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:24.064298 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:24.064702 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:24.064707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:24.064711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:24.065074 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:24.065251 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:24.065987 (+ 736us) server_negotiation.cc:299] Negotiation successful
0810 06:37:24.066264 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:37:24.067478 2137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:24.046556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:35305 (local address 127.31.59.195:57659)
0810 06:37:24.047352 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:24.047387 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:24.049289 (+ 1902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:24.050059 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:24.050073 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:24.050866 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:24.051699 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:24.051722 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.053219 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.053225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:24.055353 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:24.055366 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.056430 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.056435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:24.056613 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:24.057311 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:24.057340 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:24.060085 (+ 2745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:24.063285 (+ 3200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:24.063298 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:24.063317 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:24.063936 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:24.064408 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:24.064411 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:24.064414 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:24.064604 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:24.065200 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:24.065206 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:24.065518 (+ 312us) client_negotiation.cc:769] Sending connection context
0810 06:37:24.065818 (+ 300us) client_negotiation.cc:240] Negotiation successful
0810 06:37:24.066187 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":339,"threads_started":1}
I20260810 06:37:24.086704 2213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:24.072883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:35305 (local address 127.31.59.195:48071)
0810 06:37:24.074788 (+ 1905us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:24.074804 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:24.074904 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:24.075215 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:24.075218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:24.075617 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:24.075941 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:24.075952 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.077338 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.077342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:24.078352 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:24.078364 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.079233 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.079240 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:24.079395 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:24.079918 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:24.079934 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:24.080590 (+ 656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:24.084199 (+ 3609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:24.084209 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:24.084213 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:24.084683 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:24.085103 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:24.085106 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:24.085108 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:24.085240 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:24.085809 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:24.085814 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:24.086001 (+ 187us) client_negotiation.cc:769] Sending connection context
0810 06:37:24.086122 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 06:37:24.086382 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1789,"thread_start_us":1565,"threads_started":1}
I20260810 06:37:24.086905 1676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:24.072835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:48071 (local address 127.31.59.254:35305)
0810 06:37:24.073020 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:24.073027 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:24.074886 (+ 1859us) server_negotiation.cc:407] Connection header received
0810 06:37:24.074989 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:24.074995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:24.075079 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:24.075175 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:24.076063 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.077213 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.078487 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.079044 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.079164 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:24.080759 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:24.080764 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:24.080767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:24.080798 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:24.084055 (+ 3257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:24.084812 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:24.084817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:24.084821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:24.084992 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:24.085328 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:24.085333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:24.085337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:24.085676 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:24.085833 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:24.086268 (+ 435us) server_negotiation.cc:299] Negotiation successful
0810 06:37:24.086534 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:37:24.101567 2104 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:37041
I20260810 06:37:24.101605 2337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:37041 every 8 connection(s)
I20260810 06:37:24.102773 2104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:37:24.104756 2338 heartbeater.cc:347] Connected to a master server at 127.31.59.254:35305
I20260810 06:37:24.104971 2338 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:24.105659 2338 heartbeater.cc:511] Master 127.31.59.254:35305 requested a full tablet report, sending...
I20260810 06:37:24.106621 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 2104
I20260810 06:37:24.106783 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:37:24.107023 1500 ts_manager.cc:194] Registered new tserver with Master: 54edb32f730f45e7974a7a9068a7a906 (127.31.59.195:37041)
I20260810 06:37:24.107157 31983 external_mini_cluster.cc:1428] Setting key 46ac2258301e88fe44764121633f70f4
I20260810 06:37:24.107931 1500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:57659
I20260810 06:37:24.110179 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:37:24.131301 1849 heartbeater.cc:503] Master 127.31.59.254:35305 was elected leader, sending a full tablet report...
Aug 10 06:37:24 dist-test-slave-4tdj krb5kdc[1420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343844, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:37:24 dist-test-slave-4tdj krb5kdc[1420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343844, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:24.156921 1676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:24.143045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51404 (local address 127.31.59.254:35305)
0810 06:37:24.143261 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:24.143267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:24.143557 (+ 290us) server_negotiation.cc:407] Connection header received
0810 06:37:24.143744 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:24.143748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:24.143812 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:24.143899 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:24.145255 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.145993 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.147294 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.147717 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.147830 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:24.151845 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:24.151851 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:24.151854 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:24.151892 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:24.154391 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:24.155004 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:24.155009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:24.155014 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:24.155160 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:24.155530 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:24.155536 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:24.155540 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:24.155835 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:24.156097 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:24.156387 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 06:37:24.156605 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:37:24.160863 1500 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:37:24.167596 1500 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:37:24.190183 2360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:24.179541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:32979 (local address 127.0.0.1:39974)
0810 06:37:24.180761 (+ 1220us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:24.180808 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:24.180961 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:24.183370 (+ 2409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:24.183378 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:24.183411 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:24.183844 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:24.183859 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.185538 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.185543 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:24.187614 (+ 2071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:24.187626 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.189243 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.189249 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:24.189435 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:24.189498 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 06:37:24.189633 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 06:37:24.189812 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"thread_start_us":2019,"threads_started":1}
I20260810 06:37:24.191059 2358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:24.179932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:40139 (local address 127.0.0.1:46082)
0810 06:37:24.181857 (+ 1925us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:24.181876 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:24.182004 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:24.184210 (+ 2206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:24.184214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:24.184235 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:24.184629 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:24.184640 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.186443 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.186452 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:24.188474 (+ 2022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:24.188493 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.190250 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.190256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:24.190433 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:24.190464 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:37:24.190525 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 06:37:24.190646 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1786,"mutex_wait_us":24,"thread_start_us":697,"threads_started":1}
I20260810 06:37:24.191423 2359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:24.179683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39974 (local address 127.31.59.194:32979)
0810 06:37:24.181530 (+ 1847us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:24.181539 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:24.181571 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:37:24.181673 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:24.181682 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:24.181943 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:24.182147 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:24.184072 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.185381 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.187778 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.189051 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.189210 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:24.189437 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:24.189686 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 06:37:24.189870 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1670,"thread_start_us":1523,"threads_started":1}
I20260810 06:37:24.191859 2363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:24.181952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:37041 (local address 127.0.0.1:41616)
0810 06:37:24.182490 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:24.182507 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:24.182615 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:24.184002 (+ 1387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:24.184007 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:24.184031 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:24.184483 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:24.184499 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.186367 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.186373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:24.188915 (+ 2542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:24.188930 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.190947 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.190955 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:24.191228 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:24.191266 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 06:37:24.191344 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 06:37:24.191464 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":1,"thread_start_us":753,"threads_started":1}
I20260810 06:37:24.192237 2362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:24.180443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46082 (local address 127.31.59.193:40139)
0810 06:37:24.183682 (+ 3239us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:24.183690 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:24.183717 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:37:24.183808 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:24.183813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:24.183995 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:24.184791 (+ 796us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:24.184934 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.186246 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.188652 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.190018 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.190960 (+ 942us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:24.191411 (+ 451us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:24.191493 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 06:37:24.191806 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3099,"thread_start_us":2952,"threads_started":1}
I20260810 06:37:24.193063 2365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:24.182553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41616 (local address 127.31.59.195:37041)
0810 06:37:24.183391 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:24.183400 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:24.183436 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:37:24.183544 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:24.183553 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:24.183790 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:24.184027 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:24.184719 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.186180 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.189090 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.190628 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.192231 (+ 1603us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:24.192463 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:24.192539 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 06:37:24.192705 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":493,"threads_started":1}
I20260810 06:37:24.195370 1737 tablet_service.cc:1467] Processing CreateTablet for tablet e816cdf4da374e8c8bc39af1525517c8 (DEFAULT_TABLE table=test-table [id=61c744f5fdc8437ea71ad6a47f0969b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:24.195147 1978 tablet_service.cc:1467] Processing CreateTablet for tablet e816cdf4da374e8c8bc39af1525517c8 (DEFAULT_TABLE table=test-table [id=61c744f5fdc8437ea71ad6a47f0969b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:24.196826 1737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e816cdf4da374e8c8bc39af1525517c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:24.196287 2218 tablet_service.cc:1467] Processing CreateTablet for tablet e816cdf4da374e8c8bc39af1525517c8 (DEFAULT_TABLE table=test-table [id=61c744f5fdc8437ea71ad6a47f0969b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:24.196889 1978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e816cdf4da374e8c8bc39af1525517c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:24.197630 2218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e816cdf4da374e8c8bc39af1525517c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:24.209172 2366 tablet_bootstrap.cc:492] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8: Bootstrap starting.
I20260810 06:37:24.212451 2366 tablet_bootstrap.cc:654] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:24.213644 2367 tablet_bootstrap.cc:492] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd: Bootstrap starting.
I20260810 06:37:24.214385 2366 log.cc:824] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:24.214701 2368 tablet_bootstrap.cc:492] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906: Bootstrap starting.
I20260810 06:37:24.218027 2366 tablet_bootstrap.cc:492] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8: No bootstrap required, opened a new log
I20260810 06:37:24.218020 2367 tablet_bootstrap.cc:654] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:24.218361 2366 ts_tablet_manager.cc:1397] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260810 06:37:24.219273 2368 tablet_bootstrap.cc:654] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:24.219417 2367 log.cc:824] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:24.220788 2368 log.cc:824] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:24.222082 2367 tablet_bootstrap.cc:492] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd: No bootstrap required, opened a new log
I20260810 06:37:24.222389 2367 ts_tablet_manager.cc:1397] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.006s
I20260810 06:37:24.224270 2368 tablet_bootstrap.cc:492] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906: No bootstrap required, opened a new log
I20260810 06:37:24.224565 2368 ts_tablet_manager.cc:1397] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260810 06:37:24.230557 2367 raft_consensus.cc:348] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } }
I20260810 06:37:24.230396 2366 raft_consensus.cc:348] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } }
I20260810 06:37:24.230882 2367 raft_consensus.cc:374] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:24.230985 2367 raft_consensus.cc:729] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d740a1d4acb946028e53c02e191b30cd, State: Initialized, Role: FOLLOWER
I20260810 06:37:24.231012 2366 raft_consensus.cc:374] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:24.231191 2366 raft_consensus.cc:729] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d53289a6c73403ba1a5b37cf926fcb8, State: Initialized, Role: FOLLOWER
I20260810 06:37:24.231626 2367 consensus_queue.cc:260] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } }
I20260810 06:37:24.231822 2366 consensus_queue.cc:260] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } }
I20260810 06:37:24.233052 2094 heartbeater.cc:503] Master 127.31.59.254:35305 was elected leader, sending a full tablet report...
I20260810 06:37:24.233709 2366 ts_tablet_manager.cc:1428] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8: Time spent starting tablet: real 0.015s user 0.012s sys 0.002s
I20260810 06:37:24.235316 2367 ts_tablet_manager.cc:1428] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260810 06:37:24.237389 2368 raft_consensus.cc:348] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } }
I20260810 06:37:24.237790 2368 raft_consensus.cc:374] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:24.237895 2368 raft_consensus.cc:729] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54edb32f730f45e7974a7a9068a7a906, State: Initialized, Role: FOLLOWER
I20260810 06:37:24.238469 2368 consensus_queue.cc:260] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } }
I20260810 06:37:24.239616 2338 heartbeater.cc:503] Master 127.31.59.254:35305 was elected leader, sending a full tablet report...
I20260810 06:37:24.240161 2368 ts_tablet_manager.cc:1428] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906: Time spent starting tablet: real 0.015s user 0.008s sys 0.008s
I20260810 06:37:24.279229 2375 raft_consensus.cc:482] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:37:24.279675 2375 raft_consensus.cc:504] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } }
I20260810 06:37:24.281477 2375 leader_election.cc:288] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d53289a6c73403ba1a5b37cf926fcb8 (127.31.59.194:32979), 54edb32f730f45e7974a7a9068a7a906 (127.31.59.195:37041)
I20260810 06:37:24.288908 2359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:24.282085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:48999 (local address 127.31.59.194:32979)
0810 06:37:24.282370 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:24.282376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:24.282934 (+ 558us) server_negotiation.cc:407] Connection header received
0810 06:37:24.283219 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:24.283225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:24.283304 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:24.283368 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:24.284202 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.284915 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.287282 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.288059 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.288209 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:24.288282 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:24.288549 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 06:37:24.288673 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260810 06:37:24.289047 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:24.281963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:32979 (local address 127.31.59.193:48999)
0810 06:37:24.282826 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:24.282846 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:24.283005 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:24.283547 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:24.283550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:24.283573 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:24.284057 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:24.284067 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.285043 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.285047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:24.287124 (+ 2077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:24.287137 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.288264 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.288271 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:24.288396 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:24.288419 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 06:37:24.288500 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:37:24.288624 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"mutex_wait_us":4,"thread_start_us":516,"threads_started":1}
I20260810 06:37:24.289616 2365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:24.283069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:36865 (local address 127.31.59.195:37041)
0810 06:37:24.283282 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:24.283288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:24.283632 (+ 344us) server_negotiation.cc:407] Connection header received
0810 06:37:24.283823 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:24.283827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:24.283892 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:24.283976 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:24.284662 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.285667 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.287811 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.288637 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.288804 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:24.288887 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:24.289256 (+ 369us) server_negotiation.cc:299] Negotiation successful
0810 06:37:24.289384 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 06:37:24.289654 2381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:24.282906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:37041 (local address 127.31.59.193:36865)
0810 06:37:24.283516 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:24.283531 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:24.283678 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:24.284029 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:24.284039 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:24.284067 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:24.284470 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:24.284485 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.285815 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.285819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:24.287641 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:24.287656 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.288874 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.288886 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:24.289093 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:24.289128 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 06:37:24.289226 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 06:37:24.289358 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":290,"threads_started":1}
I20260810 06:37:24.290237 2004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e816cdf4da374e8c8bc39af1525517c8" candidate_uuid: "d740a1d4acb946028e53c02e191b30cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" is_pre_election: true
I20260810 06:37:24.290661 2256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e816cdf4da374e8c8bc39af1525517c8" candidate_uuid: "d740a1d4acb946028e53c02e191b30cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54edb32f730f45e7974a7a9068a7a906" is_pre_election: true
I20260810 06:37:24.290825 2004 raft_consensus.cc:2454] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d740a1d4acb946028e53c02e191b30cd in term 0.
I20260810 06:37:24.291136 2256 raft_consensus.cc:2454] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d740a1d4acb946028e53c02e191b30cd in term 0.
I20260810 06:37:24.291594 1656 leader_election.cc:302] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d53289a6c73403ba1a5b37cf926fcb8, d740a1d4acb946028e53c02e191b30cd; no voters:
I20260810 06:37:24.292023 2375 raft_consensus.cc:2781] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:37:24.292156 2375 raft_consensus.cc:482] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:37:24.292302 2375 raft_consensus.cc:3037] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:24.294952 2375 raft_consensus.cc:504] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } }
I20260810 06:37:24.295499 2375 leader_election.cc:288] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [CANDIDATE]: Term 1 election: Requested vote from peers 9d53289a6c73403ba1a5b37cf926fcb8 (127.31.59.194:32979), 54edb32f730f45e7974a7a9068a7a906 (127.31.59.195:37041)
I20260810 06:37:24.295967 2004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e816cdf4da374e8c8bc39af1525517c8" candidate_uuid: "d740a1d4acb946028e53c02e191b30cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d53289a6c73403ba1a5b37cf926fcb8"
I20260810 06:37:24.295984 2256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e816cdf4da374e8c8bc39af1525517c8" candidate_uuid: "d740a1d4acb946028e53c02e191b30cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54edb32f730f45e7974a7a9068a7a906"
I20260810 06:37:24.296159 2004 raft_consensus.cc:3037] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:24.296162 2256 raft_consensus.cc:3037] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:24.298592 2004 raft_consensus.cc:2454] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d740a1d4acb946028e53c02e191b30cd in term 1.
I20260810 06:37:24.298949 1656 leader_election.cc:302] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d53289a6c73403ba1a5b37cf926fcb8, d740a1d4acb946028e53c02e191b30cd; no voters:
I20260810 06:37:24.299064 2256 raft_consensus.cc:2454] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d740a1d4acb946028e53c02e191b30cd in term 1.
I20260810 06:37:24.299248 2375 raft_consensus.cc:2781] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:24.299746 2375 raft_consensus.cc:686] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [term 1 LEADER]: Becoming Leader. State: Replica: d740a1d4acb946028e53c02e191b30cd, State: Running, Role: LEADER
I20260810 06:37:24.300235 2375 consensus_queue.cc:237] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } }
I20260810 06:37:24.304977 1500 catalog_manager.cc:5495] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d740a1d4acb946028e53c02e191b30cd (127.31.59.193). New cstate: current_term: 1 leader_uuid: "d740a1d4acb946028e53c02e191b30cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } health_report { overall_health: HEALTHY } } }
W20260810 06:37:24.354571 2339 tablet.cc:2369] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:37:24.356536 2362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:24.350837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46098 (local address 127.31.59.193:40139)
0810 06:37:24.351060 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:24.351065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:24.351083 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:37:24.351373 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:24.351376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:24.351459 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:24.351609 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:37:24.351895 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.352845 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.354170 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.354625 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.354737 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:24.355427 (+ 690us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:37:24.355979 (+ 552us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:37:24.356087 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:24.356197 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 06:37:24.356261 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260810 06:37:24.358603 2098 tablet.cc:2369] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:37:24.365933 2256 raft_consensus.cc:1261] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906 [term 1 FOLLOWER]: Refusing update from remote peer d740a1d4acb946028e53c02e191b30cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:24.367129 2386 consensus_queue.cc:1035] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:24.367439 2004 raft_consensus.cc:1261] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8 [term 1 FOLLOWER]: Refusing update from remote peer d740a1d4acb946028e53c02e191b30cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:24.368180 2386 consensus_queue.cc:1035] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 06:37:24.373909 1850 tablet.cc:2369] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:37:24.381147 2389 mvcc.cc:205] Tried to move back new op lower bound from 7316864386510585856 to 7316864386257969152. Current Snapshot: MvccSnapshot[applied={T|T < 7316864386510585856}]
I20260810 06:37:24.403398 1500 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:37:24.403759 1500 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:37:24.408543 1500 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5: Sending DeleteTablet for 3 replicas of tablet e816cdf4da374e8c8bc39af1525517c8
I20260810 06:37:24.410210 1978 tablet_service.cc:1514] Processing DeleteTablet for tablet e816cdf4da374e8c8bc39af1525517c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:24 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:39974
I20260810 06:37:24.410238 1737 tablet_service.cc:1514] Processing DeleteTablet for tablet e816cdf4da374e8c8bc39af1525517c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:24 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:46082
I20260810 06:37:24.410387 2218 tablet_service.cc:1514] Processing DeleteTablet for tablet e816cdf4da374e8c8bc39af1525517c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:24 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:41616
I20260810 06:37:24.414366 1500 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:37:24.415279 1500 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:37:24.422736 2411 tablet_replica.cc:330] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906: stopping tablet replica
I20260810 06:37:24.423321 2414 tablet_replica.cc:330] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8: stopping tablet replica
I20260810 06:37:24.423725 2411 raft_consensus.cc:2229] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:37:24.423808 2414 raft_consensus.cc:2229] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:37:24.424216 2410 tablet_replica.cc:330] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd: stopping tablet replica
I20260810 06:37:24.424364 1737 tablet_service.cc:1467] Processing CreateTablet for tablet 0e8f08f1185240c49bc280f8be44b91b (DEFAULT_TABLE table=test-table [id=48136e00586c4e27a15691fcd76ca477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:24.424621 2411 raft_consensus.cc:2258] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:24.424482 1977 tablet_service.cc:1467] Processing CreateTablet for tablet 0e8f08f1185240c49bc280f8be44b91b (DEFAULT_TABLE table=test-table [id=48136e00586c4e27a15691fcd76ca477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:24.424692 2410 raft_consensus.cc:2229] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:37:24.425061 1737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e8f08f1185240c49bc280f8be44b91b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:24.425274 1977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e8f08f1185240c49bc280f8be44b91b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:24.426275 2414 raft_consensus.cc:2258] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:24.425637 2410 raft_consensus.cc:2258] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:24.428159 2217 tablet_service.cc:1467] Processing CreateTablet for tablet 0e8f08f1185240c49bc280f8be44b91b (DEFAULT_TABLE table=test-table [id=48136e00586c4e27a15691fcd76ca477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:24.428714 2217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e8f08f1185240c49bc280f8be44b91b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:24.428990 2414 ts_tablet_manager.cc:1905] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:24.430027 2411 ts_tablet_manager.cc:1905] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:24.434567 2366 tablet_bootstrap.cc:492] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8: Bootstrap starting.
I20260810 06:37:24.436713 2366 tablet_bootstrap.cc:654] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:24.437605 2414 ts_tablet_manager.cc:1918] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:37:24.437904 2414 log.cc:1196] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/e816cdf4da374e8c8bc39af1525517c8
I20260810 06:37:24.438599 2414 ts_tablet_manager.cc:1939] T e816cdf4da374e8c8bc39af1525517c8 P 9d53289a6c73403ba1a5b37cf926fcb8: Deleting consensus metadata
I20260810 06:37:24.440683 2367 tablet_bootstrap.cc:492] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd: Bootstrap starting.
I20260810 06:37:24.440991 1480 catalog_manager.cc:4841] TS 9d53289a6c73403ba1a5b37cf926fcb8 (127.31.59.194:32979): tablet e816cdf4da374e8c8bc39af1525517c8 (table test-table [id=61c744f5fdc8437ea71ad6a47f0969b5]) successfully deleted
I20260810 06:37:24.441691 2368 tablet_bootstrap.cc:492] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906: Bootstrap starting.
I20260810 06:37:24.442457 2367 tablet_bootstrap.cc:654] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:24.443351 2368 tablet_bootstrap.cc:654] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:24.443368 2366 tablet_bootstrap.cc:492] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8: No bootstrap required, opened a new log
I20260810 06:37:24.443519 2366 ts_tablet_manager.cc:1397] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.004s
I20260810 06:37:24.444396 2410 ts_tablet_manager.cc:1905] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:24.444355 2366 raft_consensus.cc:348] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } }
I20260810 06:37:24.444600 2366 raft_consensus.cc:374] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:24.444712 2366 raft_consensus.cc:729] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d53289a6c73403ba1a5b37cf926fcb8, State: Initialized, Role: FOLLOWER
I20260810 06:37:24.445051 2366 consensus_queue.cc:260] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } }
I20260810 06:37:24.445945 2366 ts_tablet_manager.cc:1428] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260810 06:37:24.450950 2411 ts_tablet_manager.cc:1918] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:37:24.451202 2411 log.cc:1196] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/e816cdf4da374e8c8bc39af1525517c8
I20260810 06:37:24.452329 2367 tablet_bootstrap.cc:492] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd: No bootstrap required, opened a new log
I20260810 06:37:24.452592 2367 ts_tablet_manager.cc:1397] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.002s
I20260810 06:37:24.452823 2411 ts_tablet_manager.cc:1939] T e816cdf4da374e8c8bc39af1525517c8 P 54edb32f730f45e7974a7a9068a7a906: Deleting consensus metadata
I20260810 06:37:24.454064 2367 raft_consensus.cc:348] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } }
I20260810 06:37:24.454403 2367 raft_consensus.cc:374] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:24.454630 2367 raft_consensus.cc:729] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d740a1d4acb946028e53c02e191b30cd, State: Initialized, Role: FOLLOWER
I20260810 06:37:24.454843 2368 tablet_bootstrap.cc:492] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906: No bootstrap required, opened a new log
I20260810 06:37:24.455092 2368 ts_tablet_manager.cc:1397] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.007s
I20260810 06:37:24.455080 1480 catalog_manager.cc:4841] TS 54edb32f730f45e7974a7a9068a7a906 (127.31.59.195:37041): tablet e816cdf4da374e8c8bc39af1525517c8 (table test-table [id=61c744f5fdc8437ea71ad6a47f0969b5]) successfully deleted
I20260810 06:37:24.456539 2368 raft_consensus.cc:348] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } }
I20260810 06:37:24.456799 2368 raft_consensus.cc:374] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:24.456971 2368 raft_consensus.cc:729] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54edb32f730f45e7974a7a9068a7a906, State: Initialized, Role: FOLLOWER
I20260810 06:37:24.457140 2367 consensus_queue.cc:260] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } }
I20260810 06:37:24.457351 2368 consensus_queue.cc:260] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } }
I20260810 06:37:24.458256 2367 ts_tablet_manager.cc:1428] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260810 06:37:24.460350 2368 ts_tablet_manager.cc:1428] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260810 06:37:24.463084 2410 ts_tablet_manager.cc:1918] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:37:24.463348 2410 log.cc:1196] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/e816cdf4da374e8c8bc39af1525517c8
I20260810 06:37:24.464121 2410 ts_tablet_manager.cc:1939] T e816cdf4da374e8c8bc39af1525517c8 P d740a1d4acb946028e53c02e191b30cd: Deleting consensus metadata
I20260810 06:37:24.465826 1474 catalog_manager.cc:4841] TS d740a1d4acb946028e53c02e191b30cd (127.31.59.193:40139): tablet e816cdf4da374e8c8bc39af1525517c8 (table test-table [id=61c744f5fdc8437ea71ad6a47f0969b5]) successfully deleted
I20260810 06:37:24.589931 2375 raft_consensus.cc:482] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:37:24.590207 2375 raft_consensus.cc:504] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } }
I20260810 06:37:24.591063 2375 leader_election.cc:288] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d53289a6c73403ba1a5b37cf926fcb8 (127.31.59.194:32979), 54edb32f730f45e7974a7a9068a7a906 (127.31.59.195:37041)
I20260810 06:37:24.591727 2002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e8f08f1185240c49bc280f8be44b91b" candidate_uuid: "d740a1d4acb946028e53c02e191b30cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" is_pre_election: true
I20260810 06:37:24.591801 2256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e8f08f1185240c49bc280f8be44b91b" candidate_uuid: "d740a1d4acb946028e53c02e191b30cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54edb32f730f45e7974a7a9068a7a906" is_pre_election: true
I20260810 06:37:24.592026 2002 raft_consensus.cc:2454] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d740a1d4acb946028e53c02e191b30cd in term 0.
I20260810 06:37:24.592162 2256 raft_consensus.cc:2454] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d740a1d4acb946028e53c02e191b30cd in term 0.
I20260810 06:37:24.592608 1656 leader_election.cc:302] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d53289a6c73403ba1a5b37cf926fcb8, d740a1d4acb946028e53c02e191b30cd; no voters:
I20260810 06:37:24.593156 2375 raft_consensus.cc:2781] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:37:24.593330 2375 raft_consensus.cc:482] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:37:24.593525 2375 raft_consensus.cc:3037] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:24.596769 2375 raft_consensus.cc:504] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } }
I20260810 06:37:24.597615 2375 leader_election.cc:288] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [CANDIDATE]: Term 1 election: Requested vote from peers 9d53289a6c73403ba1a5b37cf926fcb8 (127.31.59.194:32979), 54edb32f730f45e7974a7a9068a7a906 (127.31.59.195:37041)
I20260810 06:37:24.597914 2002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e8f08f1185240c49bc280f8be44b91b" candidate_uuid: "d740a1d4acb946028e53c02e191b30cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d53289a6c73403ba1a5b37cf926fcb8"
I20260810 06:37:24.598258 2002 raft_consensus.cc:3037] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:24.598253 2256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e8f08f1185240c49bc280f8be44b91b" candidate_uuid: "d740a1d4acb946028e53c02e191b30cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54edb32f730f45e7974a7a9068a7a906"
I20260810 06:37:24.598513 2256 raft_consensus.cc:3037] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:24.601361 2256 raft_consensus.cc:2454] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d740a1d4acb946028e53c02e191b30cd in term 1.
I20260810 06:37:24.601418 2002 raft_consensus.cc:2454] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d740a1d4acb946028e53c02e191b30cd in term 1.
I20260810 06:37:24.601850 1656 leader_election.cc:302] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d53289a6c73403ba1a5b37cf926fcb8, d740a1d4acb946028e53c02e191b30cd; no voters:
I20260810 06:37:24.602149 2375 raft_consensus.cc:2781] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:24.602362 2375 raft_consensus.cc:686] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [term 1 LEADER]: Becoming Leader. State: Replica: d740a1d4acb946028e53c02e191b30cd, State: Running, Role: LEADER
I20260810 06:37:24.602648 2375 consensus_queue.cc:237] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } }
I20260810 06:37:24.605777 1498 catalog_manager.cc:5495] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d740a1d4acb946028e53c02e191b30cd (127.31.59.193). New cstate: current_term: 1 leader_uuid: "d740a1d4acb946028e53c02e191b30cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } health_report { overall_health: UNKNOWN } } }
Aug 10 06:37:24 dist-test-slave-4tdj krb5kdc[1420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343842, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:24.679380 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:24.664562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51414 (local address 127.31.59.254:35305)
0810 06:37:24.665654 (+ 1092us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:24.665658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:24.665677 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:37:24.665723 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:24.665726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:24.665780 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:24.665871 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:24.667324 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.668192 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.669528 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.669973 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.670085 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:24.673661 (+ 3576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:24.673666 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:24.673669 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:24.673698 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:24.676360 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:24.677262 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:24.677266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:24.677268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:24.677397 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:24.677822 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:24.677826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:24.677828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:24.678094 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:24.678212 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:24.678754 (+ 542us) server_negotiation.cc:299] Negotiation successful
0810 06:37:24.679059 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":123,"server-negotiator.queue_time_us":984,"thread_start_us":771,"threads_started":1}
I20260810 06:37:24.679458 2430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:24.664449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:35305 (local address 127.0.0.1:51414)
0810 06:37:24.665093 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:24.665123 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:24.665298 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:24.665929 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:24.665935 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:24.666803 (+ 868us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:24.667160 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:24.667173 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.668315 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.668319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:24.669369 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:24.669383 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:24.670163 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:24.670169 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:24.670370 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:24.671033 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:24.671050 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:24.673437 (+ 2387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:24.676509 (+ 3072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:24.676522 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:24.676530 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:24.677152 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:24.677504 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:24.677509 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:24.677513 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:24.677688 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:24.678231 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:24.678238 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:24.678463 (+ 225us) client_negotiation.cc:769] Sending connection context
0810 06:37:24.678651 (+ 188us) client_negotiation.cc:240] Negotiation successful
0810 06:37:24.678939 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":304,"threads_started":1}
I20260810 06:37:24.685004 1498 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:51414:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 06:37:24.686659 1498 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:37:24.694123 1737 tablet_service.cc:1467] Processing CreateTablet for tablet c95f6f10146d4f85a8ea2266b320e73e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c5e54fcd7f0540878c488f7437cc163d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:37:24.694201 1977 tablet_service.cc:1467] Processing CreateTablet for tablet c95f6f10146d4f85a8ea2266b320e73e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c5e54fcd7f0540878c488f7437cc163d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:37:24.694430 2217 tablet_service.cc:1467] Processing CreateTablet for tablet c95f6f10146d4f85a8ea2266b320e73e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c5e54fcd7f0540878c488f7437cc163d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:37:24.694733 1737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c95f6f10146d4f85a8ea2266b320e73e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:24.694762 1977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c95f6f10146d4f85a8ea2266b320e73e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:24.694945 2217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c95f6f10146d4f85a8ea2266b320e73e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:24.702785 2368 tablet_bootstrap.cc:492] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906: Bootstrap starting.
I20260810 06:37:24.704880 2368 tablet_bootstrap.cc:654] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:24.705569 2366 tablet_bootstrap.cc:492] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8: Bootstrap starting.
I20260810 06:37:24.706369 2367 tablet_bootstrap.cc:492] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd: Bootstrap starting.
I20260810 06:37:24.707378 2366 tablet_bootstrap.cc:654] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:24.708263 2367 tablet_bootstrap.cc:654] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:24.717878 2366 tablet_bootstrap.cc:492] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8: No bootstrap required, opened a new log
I20260810 06:37:24.718056 2366 ts_tablet_manager.cc:1397] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.008s
I20260810 06:37:24.718746 2368 tablet_bootstrap.cc:492] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906: No bootstrap required, opened a new log
I20260810 06:37:24.718931 2368 ts_tablet_manager.cc:1397] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.010s
I20260810 06:37:24.718999 2366 raft_consensus.cc:348] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } }
I20260810 06:37:24.719264 2366 raft_consensus.cc:374] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:24.719350 2366 raft_consensus.cc:729] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d53289a6c73403ba1a5b37cf926fcb8, State: Initialized, Role: FOLLOWER
I20260810 06:37:24.719732 2367 tablet_bootstrap.cc:492] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd: No bootstrap required, opened a new log
I20260810 06:37:24.719645 2366 consensus_queue.cc:260] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } }
I20260810 06:37:24.719686 2368 raft_consensus.cc:348] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } }
I20260810 06:37:24.719915 2368 raft_consensus.cc:374] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:24.720041 2368 raft_consensus.cc:729] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54edb32f730f45e7974a7a9068a7a906, State: Initialized, Role: FOLLOWER
I20260810 06:37:24.720152 2367 ts_tablet_manager.cc:1397] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.003s
I20260810 06:37:24.720320 2368 consensus_queue.cc:260] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } }
I20260810 06:37:24.720191 2376 tablet_replica.cc:439] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } }
I20260810 06:37:24.720768 2376 tablet_replica.cc:442] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:37:24.720837 2378 tablet_replica.cc:439] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } }
I20260810 06:37:24.721352 2378 tablet_replica.cc:442] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:37:24.722625 2367 raft_consensus.cc:348] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } }
I20260810 06:37:24.722959 2367 raft_consensus.cc:374] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:24.723135 2367 raft_consensus.cc:729] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d740a1d4acb946028e53c02e191b30cd, State: Initialized, Role: FOLLOWER
I20260810 06:37:24.723611 2366 ts_tablet_manager.cc:1428] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260810 06:37:24.724076 2368 ts_tablet_manager.cc:1428] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260810 06:37:24.723600 2367 consensus_queue.cc:260] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } }
I20260810 06:37:24.724819 2375 tablet_replica.cc:439] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } }
I20260810 06:37:24.725240 2375 tablet_replica.cc:442] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:37:24.726620 2367 ts_tablet_manager.cc:1428] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260810 06:37:24.945238 2375 raft_consensus.cc:482] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:37:24.945443 2375 raft_consensus.cc:504] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } }
I20260810 06:37:24.946177 2375 leader_election.cc:288] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d53289a6c73403ba1a5b37cf926fcb8 (127.31.59.194:32979), 54edb32f730f45e7974a7a9068a7a906 (127.31.59.195:37041)
I20260810 06:37:24.946749 2002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c95f6f10146d4f85a8ea2266b320e73e" candidate_uuid: "d740a1d4acb946028e53c02e191b30cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" is_pre_election: true
I20260810 06:37:24.946888 2256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c95f6f10146d4f85a8ea2266b320e73e" candidate_uuid: "d740a1d4acb946028e53c02e191b30cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54edb32f730f45e7974a7a9068a7a906" is_pre_election: true
I20260810 06:37:24.947038 2002 raft_consensus.cc:2454] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d740a1d4acb946028e53c02e191b30cd in term 0.
I20260810 06:37:24.947157 2256 raft_consensus.cc:2454] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d740a1d4acb946028e53c02e191b30cd in term 0.
I20260810 06:37:24.947499 1656 leader_election.cc:302] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d53289a6c73403ba1a5b37cf926fcb8, d740a1d4acb946028e53c02e191b30cd; no voters:
I20260810 06:37:24.947783 2375 raft_consensus.cc:2781] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:37:24.947890 2375 raft_consensus.cc:482] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:37:24.947955 2375 raft_consensus.cc:3037] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:24.949887 2375 raft_consensus.cc:504] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } }
I20260810 06:37:24.950371 2375 leader_election.cc:288] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [CANDIDATE]: Term 1 election: Requested vote from peers 9d53289a6c73403ba1a5b37cf926fcb8 (127.31.59.194:32979), 54edb32f730f45e7974a7a9068a7a906 (127.31.59.195:37041)
I20260810 06:37:24.950857 2002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c95f6f10146d4f85a8ea2266b320e73e" candidate_uuid: "d740a1d4acb946028e53c02e191b30cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d53289a6c73403ba1a5b37cf926fcb8"
I20260810 06:37:24.950871 2256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c95f6f10146d4f85a8ea2266b320e73e" candidate_uuid: "d740a1d4acb946028e53c02e191b30cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54edb32f730f45e7974a7a9068a7a906"
I20260810 06:37:24.951084 2256 raft_consensus.cc:3037] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:24.951175 2002 raft_consensus.cc:3037] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:24.953351 2256 raft_consensus.cc:2454] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d740a1d4acb946028e53c02e191b30cd in term 1.
I20260810 06:37:24.953557 2002 raft_consensus.cc:2454] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d740a1d4acb946028e53c02e191b30cd in term 1.
I20260810 06:37:24.953763 1656 leader_election.cc:302] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54edb32f730f45e7974a7a9068a7a906, d740a1d4acb946028e53c02e191b30cd; no voters:
I20260810 06:37:24.954128 2375 raft_consensus.cc:2781] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:24.954274 2375 raft_consensus.cc:686] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [term 1 LEADER]: Becoming Leader. State: Replica: d740a1d4acb946028e53c02e191b30cd, State: Running, Role: LEADER
I20260810 06:37:24.954598 2375 consensus_queue.cc:237] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } }
I20260810 06:37:24.956310 2386 tablet_replica.cc:439] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd: TxnStatusTablet state changed. Reason: New leader d740a1d4acb946028e53c02e191b30cd. Latest consensus state: current_term: 1 leader_uuid: "d740a1d4acb946028e53c02e191b30cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } }
I20260810 06:37:24.956543 2386 tablet_replica.cc:442] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:37:24.957235 2439 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 06:37:24.957567 2439 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 06:37:24.957895 1500 catalog_manager.cc:5495] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d740a1d4acb946028e53c02e191b30cd (127.31.59.193). New cstate: current_term: 1 leader_uuid: "d740a1d4acb946028e53c02e191b30cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } health_report { overall_health: UNKNOWN } } }
I20260810 06:37:25.005973 2386 consensus_queue.cc:1035] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:25.031829 2446 consensus_queue.cc:1035] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:25.038319 2430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:25.025612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:40139 (local address 127.0.0.1:46100)
0810 06:37:25.025887 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:25.025910 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:25.026159 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:25.031213 (+ 5054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:25.031218 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:25.031233 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:37:25.031559 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:25.031574 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:25.033642 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:25.033648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:25.035306 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:25.035323 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:25.036668 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:25.036674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:25.036859 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:25.036936 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:37:25.037515 (+ 579us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:37:25.037541 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 06:37:25.037605 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 06:37:25.037800 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 06:37:25.039199 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:25.019612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46100 (local address 127.31.59.193:40139)
0810 06:37:25.030843 (+ 11231us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:25.030849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:25.030867 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:37:25.030932 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:25.030937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:25.031021 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:25.031712 (+ 691us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:37:25.031999 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:25.033488 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:25.035486 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:25.036493 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:25.036621 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:25.037228 (+ 607us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:37:25.037383 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:37:25.038622 (+ 1239us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:25.038687 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 06:37:25.038766 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11092,"thread_start_us":10953,"threads_started":1}
I20260810 06:37:25.043398 2002 raft_consensus.cc:1261] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8 [term 1 FOLLOWER]: Refusing update from remote peer d740a1d4acb946028e53c02e191b30cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:25.043398 2256 raft_consensus.cc:1261] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906 [term 1 FOLLOWER]: Refusing update from remote peer d740a1d4acb946028e53c02e191b30cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:25.043979 2386 consensus_queue.cc:1035] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:25.044235 2446 consensus_queue.cc:1035] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:25.047122 2378 tablet_replica.cc:439] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906: TxnStatusTablet state changed. Reason: New leader d740a1d4acb946028e53c02e191b30cd. Latest consensus state: current_term: 1 leader_uuid: "d740a1d4acb946028e53c02e191b30cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } }
I20260810 06:37:25.047365 2378 tablet_replica.cc:442] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:37:25.055509 2450 mvcc.cc:205] Tried to move back new op lower bound from 7316864389291921408 to 7316864388936384512. Current Snapshot: MvccSnapshot[applied={T|T < 7316864389291921408}]
I20260810 06:37:25.057727 2376 tablet_replica.cc:439] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8: TxnStatusTablet state changed. Reason: New leader d740a1d4acb946028e53c02e191b30cd. Latest consensus state: current_term: 1 leader_uuid: "d740a1d4acb946028e53c02e191b30cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } }
I20260810 06:37:25.058015 2376 tablet_replica.cc:442] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:37:25.061638 2443 mvcc.cc:205] Tried to move back new op lower bound from 7316864389291921408 to 7316864388936384512. Current Snapshot: MvccSnapshot[applied={T|T < 7316864389291921408}]
I20260810 06:37:25.062175 2375 tablet_replica.cc:439] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d740a1d4acb946028e53c02e191b30cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } }
I20260810 06:37:25.062497 2375 tablet_replica.cc:442] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:37:25.065243 2375 tablet_replica.cc:439] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d740a1d4acb946028e53c02e191b30cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } }
I20260810 06:37:25.065630 2375 tablet_replica.cc:442] T c95f6f10146d4f85a8ea2266b320e73e P d740a1d4acb946028e53c02e191b30cd: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:37:25.068589 2378 tablet_replica.cc:439] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d740a1d4acb946028e53c02e191b30cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } }
I20260810 06:37:25.068882 2378 tablet_replica.cc:442] T c95f6f10146d4f85a8ea2266b320e73e P 54edb32f730f45e7974a7a9068a7a906: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:37:25.072847 2376 tablet_replica.cc:439] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d740a1d4acb946028e53c02e191b30cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d740a1d4acb946028e53c02e191b30cd" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40139 } } peers { permanent_uuid: "9d53289a6c73403ba1a5b37cf926fcb8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 32979 } } peers { permanent_uuid: "54edb32f730f45e7974a7a9068a7a906" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37041 } } }
I20260810 06:37:25.073055 2376 tablet_replica.cc:442] T c95f6f10146d4f85a8ea2266b320e73e P 9d53289a6c73403ba1a5b37cf926fcb8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:37:25.087064 2473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:25.079919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:40139 (local address 127.31.59.193:49995)
0810 06:37:25.080961 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:25.080977 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:25.081099 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:25.081415 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:25.081418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:25.081431 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:37:25.081807 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:25.081817 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:25.083270 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:25.083273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:25.084840 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:25.084852 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:25.085832 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:25.085837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:25.085962 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:25.086020 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:37:25.086446 (+ 426us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:37:25.086455 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 06:37:25.086518 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 06:37:25.086616 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"thread_start_us":750,"threads_started":1}
I20260810 06:37:25.087146 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:25.079727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:49995 (local address 127.31.59.193:40139)
0810 06:37:25.079896 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:25.079902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:25.081057 (+ 1155us) server_negotiation.cc:407] Connection header received
0810 06:37:25.081208 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:25.081214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:25.081298 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:25.081386 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:37:25.081961 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:25.083149 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:25.084977 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:25.085662 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:25.085783 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:25.086197 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:37:25.086349 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:37:25.086443 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:25.086648 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 06:37:25.086730 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:37:25.175925 1498 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:37:25.176196 1498 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:37:25.179296 1498 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ccb6530327534bb791e1e5b7c4c5efd5: Sending DeleteTablet for 3 replicas of tablet 0e8f08f1185240c49bc280f8be44b91b
I20260810 06:37:25.180173 1977 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e8f08f1185240c49bc280f8be44b91b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:25 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:39974
I20260810 06:37:25.180217 1737 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e8f08f1185240c49bc280f8be44b91b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:25 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:46082
I20260810 06:37:25.180896 2217 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e8f08f1185240c49bc280f8be44b91b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:25 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:41616
I20260810 06:37:25.201972 2478 tablet_replica.cc:330] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906: stopping tablet replica
I20260810 06:37:25.202248 2478 raft_consensus.cc:2229] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:37:25.202579 2478 raft_consensus.cc:2258] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:25.203985 2478 ts_tablet_manager.cc:1905] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:25.205902 2476 tablet_replica.cc:330] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8: stopping tablet replica
I20260810 06:37:25.206091 2476 raft_consensus.cc:2229] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:37:25.206373 2476 raft_consensus.cc:2258] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:25.207261 2477 tablet_replica.cc:330] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd: stopping tablet replica
I20260810 06:37:25.207526 2477 raft_consensus.cc:2229] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:37:25.208017 2477 raft_consensus.cc:2258] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:25.209110 2477 ts_tablet_manager.cc:1905] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:25.213424 2478 ts_tablet_manager.cc:1918] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:37:25.213593 2478 log.cc:1196] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/0e8f08f1185240c49bc280f8be44b91b
I20260810 06:37:25.214046 2478 ts_tablet_manager.cc:1939] T 0e8f08f1185240c49bc280f8be44b91b P 54edb32f730f45e7974a7a9068a7a906: Deleting consensus metadata
I20260810 06:37:25.215034 2476 ts_tablet_manager.cc:1905] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:25.217276 2477 ts_tablet_manager.cc:1918] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:37:25.217538 2477 log.cc:1196] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/0e8f08f1185240c49bc280f8be44b91b
I20260810 06:37:25.218387 2477 ts_tablet_manager.cc:1939] T 0e8f08f1185240c49bc280f8be44b91b P d740a1d4acb946028e53c02e191b30cd: Deleting consensus metadata
I20260810 06:37:25.219887 1474 catalog_manager.cc:4841] TS d740a1d4acb946028e53c02e191b30cd (127.31.59.193:40139): tablet 0e8f08f1185240c49bc280f8be44b91b (table test-table [id=48136e00586c4e27a15691fcd76ca477]) successfully deleted
I20260810 06:37:25.221635 2476 ts_tablet_manager.cc:1918] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:37:25.221820 2476 log.cc:1196] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/0e8f08f1185240c49bc280f8be44b91b
I20260810 06:37:25.223379 2476 ts_tablet_manager.cc:1939] T 0e8f08f1185240c49bc280f8be44b91b P 9d53289a6c73403ba1a5b37cf926fcb8: Deleting consensus metadata
I20260810 06:37:25.223428 1480 catalog_manager.cc:4841] TS 54edb32f730f45e7974a7a9068a7a906 (127.31.59.195:37041): tablet 0e8f08f1185240c49bc280f8be44b91b (table test-table [id=48136e00586c4e27a15691fcd76ca477]) successfully deleted
I20260810 06:37:25.224407 1480 catalog_manager.cc:4841] TS 9d53289a6c73403ba1a5b37cf926fcb8 (127.31.59.194:32979): tablet 0e8f08f1185240c49bc280f8be44b91b (table test-table [id=48136e00586c4e27a15691fcd76ca477]) successfully deleted
Aug 10 06:37:25 dist-test-slave-4tdj krb5kdc[1420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343844, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:37:25.241798 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:25.223549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46104 (local address 127.31.59.193:40139)
0810 06:37:25.223728 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:25.223734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:25.224899 (+ 1165us) server_negotiation.cc:407] Connection header received
0810 06:37:25.225075 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:25.225081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:25.225177 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:25.225290 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:25.226916 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:25.227943 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:25.229186 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:25.229625 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:25.229731 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:25.234833 (+ 5102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:25.234847 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:25.234864 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:25.234897 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:25.238179 (+ 3282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:25.238959 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:25.238967 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:25.238969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:25.239234 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:25.239655 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:25.239659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:25.239661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:25.240084 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:25.240346 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:25.241338 (+ 992us) server_negotiation.cc:299] Negotiation successful
0810 06:37:25.241534 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 06:37:25.242594 1840 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46104
I20260810 06:37:25.259943 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:25.245311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51416 (local address 127.31.59.254:35305)
0810 06:37:25.245886 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:25.245891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:25.245909 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:37:25.245958 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:25.245961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:25.246028 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:25.246110 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:25.247945 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:25.248809 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:25.250471 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:25.250969 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:25.251080 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:25.254020 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:25.254024 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:25.254028 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:25.254073 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:25.257319 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:25.257985 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:25.257989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:25.257991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:25.258115 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:25.258528 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:25.258533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:25.258537 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:25.258802 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:25.259081 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:25.259401 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 06:37:25.259620 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":369,"threads_started":1}
W20260810 06:37:25.260644 1498 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51416
I20260810 06:37:25.262744 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 1627
I20260810 06:37:25.275626 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:25.263527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46106 (local address 127.31.59.193:40139)
0810 06:37:25.263681 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:25.263685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:25.264073 (+ 388us) server_negotiation.cc:407] Connection header received
0810 06:37:25.264225 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:25.264231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:25.264300 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:25.264381 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:25.265420 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:25.266151 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:25.267352 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:25.267915 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:25.268035 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:25.270233 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:25.270238 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:25.270241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:25.270279 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:25.273053 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:25.273630 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:25.273636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:25.273640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:25.273781 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:25.274130 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:25.274135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:25.274139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:25.274488 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:25.274759 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:25.275068 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 06:37:25.275303 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260810 06:37:25.276197 1840 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46106
W20260810 06:37:25.276760 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 1627: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:37:25.276918 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 1627
I20260810 06:37:25.309022 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 1856
Aug 10 06:37:25 dist-test-slave-4tdj krb5kdc[1420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343844, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.194@KRBTEST.COM
I20260810 06:37:25.323973 2488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:25.309731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39978 (local address 127.31.59.194:32979)
0810 06:37:25.310182 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:25.310187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:25.310203 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:37:25.310259 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:25.310263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:25.310369 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:25.310454 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:25.311601 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:25.312301 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:25.313597 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:25.314018 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:25.314114 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:25.318041 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:25.318057 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:25.318081 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:25.318122 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:25.320874 (+ 2752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:25.321710 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:25.321719 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:25.321721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:25.321885 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:25.322443 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:25.322447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:25.322449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:25.322838 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:25.323081 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:25.323429 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 06:37:25.323632 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":257,"threads_started":1}
W20260810 06:37:25.324738 2088 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39978
W20260810 06:37:25.325352 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 1856: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:37:25.325578 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 1856
I20260810 06:37:25.351054 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 2104
Aug 10 06:37:25 dist-test-slave-4tdj krb5kdc[1420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343844, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.195@KRBTEST.COM
I20260810 06:37:25.366215 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:25.351852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41624 (local address 127.31.59.195:37041)
0810 06:37:25.352370 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:25.352375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:25.352390 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:37:25.352442 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:25.352446 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:25.352515 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:25.352619 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:25.353884 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:25.354648 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:25.356014 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:25.356432 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:25.356522 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:25.360865 (+ 4343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:25.360880 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:25.360896 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:25.360929 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:25.363606 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:25.364233 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:25.364241 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:25.364243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:25.364390 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:25.364756 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:25.364759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:25.364761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:25.365102 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:25.365405 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:25.365678 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 06:37:25.365891 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":287,"threads_started":1}
W20260810 06:37:25.366998 2329 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41624
W20260810 06:37:25.367626 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 2104: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:37:25.367751 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 2104
I20260810 06:37:25.388206 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 1442
I20260810 06:37:25.389290 1573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:25.501478 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 1442
2026-08-10T06:37:25Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4804 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:37:25 dist-test-slave-4tdj krb5kdc[2495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:37:25 dist-test-slave-4tdj krb5kdc[2495](info): set up 2 sockets
Aug 10 06:37:25 dist-test-slave-4tdj krb5kdc[2495](info): commencing operation
krb5kdc: starting...
W20260810 06:37:26.752565 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.002s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260810 06:37:26.899833 31983 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.147s user 0.000s sys 0.007s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260810 06:37:27.093483 31983 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.193s user 0.001s sys 0.005s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260810 06:37:27.231952 31983 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.138s user 0.000s sys 0.006s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/test-user.keytab.
W20260810 06:37:27.701915 31983 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.470s user 0.000s sys 0.006s
Aug 10 06:37:27 dist-test-slave-4tdj krb5kdc[2495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:37:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:37:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
W20260810 06:37:27.890131 31983 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.31.59.254: real 0.139s user 0.005s sys 0.005s
I20260810 06:37:27.891155 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:38471
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:35445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:38471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:28.205312 2518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:28.205701 2518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:28.205852 2518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:28.212963 2518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:37:28.213085 2518 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 06:37:28.213152 2518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:28.213238 2518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:28.213304 2518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:37:28.213389 2518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:37:28.223196 2518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:35445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:38471
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:38471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:28.224468 2518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:28.225793 2518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:28.236845 2527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:28.236917 2529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:28.236879 2534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:28.237854 2518 server_base.cc:1034] running on GCE node
I20260810 06:37:28.238655 2518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:28.239935 2518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:28.241142 2518 hybrid_clock.cc:648] HybridClock initialized: now 1786343848241081 us; error 37 us; skew 500 ppm
I20260810 06:37:28.241446 2518 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:28 dist-test-slave-4tdj krb5kdc[2495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343848, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:28.248204 2518 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:37:28.249677 2518 webserver.cc:460] Webserver started at http://127.31.59.254:32819/ using document root <none> and password file <none>
I20260810 06:37:28.250352 2518 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:28.250447 2518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:28.250716 2518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:28.253516 2518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "55d6012986854c618d7029c11e126db9"
format_stamp: "Formatted at 2026-08-10 06:37:28 on dist-test-slave-4tdj"
server_key: "114fc7abde51c184d4b3f3b7131a5130"
server_key_iv: "5964914039e6ad7d14a2616be31db80d"
server_key_version: "encryptionkey@0"
I20260810 06:37:28.254223 2518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "55d6012986854c618d7029c11e126db9"
format_stamp: "Formatted at 2026-08-10 06:37:28 on dist-test-slave-4tdj"
server_key: "114fc7abde51c184d4b3f3b7131a5130"
server_key_iv: "5964914039e6ad7d14a2616be31db80d"
server_key_version: "encryptionkey@0"
I20260810 06:37:28.259068 2518 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 06:37:28.262276 2545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:28.263533 2518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:37:28.263682 2518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "55d6012986854c618d7029c11e126db9"
format_stamp: "Formatted at 2026-08-10 06:37:28 on dist-test-slave-4tdj"
server_key: "114fc7abde51c184d4b3f3b7131a5130"
server_key_iv: "5964914039e6ad7d14a2616be31db80d"
server_key_version: "encryptionkey@0"
I20260810 06:37:28.263851 2518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:28.275065 2518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:28.278931 2518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:28.279136 2518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:28.298588 2518 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:38471
I20260810 06:37:28.298646 2670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:38471 every 8 connection(s)
I20260810 06:37:28.299844 2518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:37:28.303694 2672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:28.307432 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 2518
I20260810 06:37:28.307657 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:37:28.308465 31983 external_mini_cluster.cc:1428] Setting key 3b65ed81f47bebaefe99d99d39307b1a
I20260810 06:37:28.316581 2672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9: Bootstrap starting.
I20260810 06:37:28.320361 2672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:28.321653 2672 log.cc:824] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9: Log is configured to *not* fsync() on all Append() calls
Aug 10 06:37:28 dist-test-slave-4tdj krb5kdc[2495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:28.324895 2672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9: No bootstrap required, opened a new log
I20260810 06:37:28.331215 2677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:28.310430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42856 (local address 127.31.59.254:38471)
0810 06:37:28.311113 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:28.311123 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:28.311164 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:37:28.312691 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:28.312714 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:28.313111 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:28.313427 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:28.315141 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:28.316402 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:28.318109 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:28.318832 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:28.318953 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:28.323942 (+ 4989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:28.323950 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:28.323964 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:28.324001 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:28.327000 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:28.327830 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:28.327837 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:28.327847 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:28.328027 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:28.328439 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:28.328442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:28.328444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:28.328995 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:28.329202 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:28.329749 (+ 547us) server_negotiation.cc:299] Negotiation successful
0810 06:37:28.330178 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":313,"threads_started":1}
I20260810 06:37:28.334438 2672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d6012986854c618d7029c11e126db9" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38471 } }
I20260810 06:37:28.334803 2672 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:28.334914 2672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55d6012986854c618d7029c11e126db9, State: Initialized, Role: FOLLOWER
I20260810 06:37:28.335457 2672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d6012986854c618d7029c11e126db9" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38471 } }
I20260810 06:37:28.335719 2672 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:37:28.335852 2672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:37:28.336020 2672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:28.338368 2672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d6012986854c618d7029c11e126db9" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38471 } }
I20260810 06:37:28.338874 2672 leader_election.cc:302] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55d6012986854c618d7029c11e126db9; no voters:
I20260810 06:37:28.339424 2672 leader_election.cc:288] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:37:28.339530 2681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:28.340204 2681 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9 [term 1 LEADER]: Becoming Leader. State: Replica: 55d6012986854c618d7029c11e126db9, State: Running, Role: LEADER
I20260810 06:37:28.340695 2681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d6012986854c618d7029c11e126db9" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38471 } }
I20260810 06:37:28.341492 2672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:37:28.343361 2682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55d6012986854c618d7029c11e126db9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d6012986854c618d7029c11e126db9" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38471 } } }
I20260810 06:37:28.343851 2682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:28.346257 2683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55d6012986854c618d7029c11e126db9. Latest consensus state: current_term: 1 leader_uuid: "55d6012986854c618d7029c11e126db9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d6012986854c618d7029c11e126db9" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38471 } } }
I20260810 06:37:28.346526 2683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:28.347082 2698 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:37:28.351325 2698 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:37:28.359236 2698 catalog_manager.cc:1347] Generated new cluster ID: e93bd81a26084a7cbdf87afa9aaa4a3d
I20260810 06:37:28.359338 2698 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:37:28.379386 2698 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:37:28.381215 2698 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:37:28.389940 2698 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9: Generated new TSK 0
I20260810 06:37:28.390946 2698 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
I20260810 06:37:28.442253 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:38471
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:35445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:28.759518 2723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:28.759783 2723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:28.759927 2723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:28.767055 2723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:28.767151 2723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:28.767269 2723 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 06:37:28.767330 2723 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:37:28.767426 2723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:37:28.776983 2723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:35445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.59.254:38471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:28.778170 2723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:28.779455 2723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:28.792044 2736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:28.792112 2738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:28.792133 2735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:28.792905 2723 server_base.cc:1034] running on GCE node
I20260810 06:37:28.793520 2723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:28.794405 2723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:28.795609 2723 hybrid_clock.cc:648] HybridClock initialized: now 1786343848795556 us; error 45 us; skew 500 ppm
I20260810 06:37:28.795900 2723 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:28 dist-test-slave-4tdj krb5kdc[2495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343848, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:28.798861 2723 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:37:28.800199 2723 webserver.cc:460] Webserver started at http://127.31.59.193:42849/ using document root <none> and password file <none>
I20260810 06:37:28.800834 2723 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:28.800949 2723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:28.801241 2723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:28.804054 2723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "e4125072468246afa375da1a22c5be30"
format_stamp: "Formatted at 2026-08-10 06:37:28 on dist-test-slave-4tdj"
server_key: "9ba3a45513f08f89fde6af05bf3b9075"
server_key_iv: "97431d0b35af55f1dbd82e54e6b5232c"
server_key_version: "encryptionkey@0"
I20260810 06:37:28.804853 2723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "e4125072468246afa375da1a22c5be30"
format_stamp: "Formatted at 2026-08-10 06:37:28 on dist-test-slave-4tdj"
server_key: "9ba3a45513f08f89fde6af05bf3b9075"
server_key_iv: "97431d0b35af55f1dbd82e54e6b5232c"
server_key_version: "encryptionkey@0"
I20260810 06:37:28.809742 2723 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:37:28.813167 2749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:28.814445 2723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 06:37:28.814623 2723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "e4125072468246afa375da1a22c5be30"
format_stamp: "Formatted at 2026-08-10 06:37:28 on dist-test-slave-4tdj"
server_key: "9ba3a45513f08f89fde6af05bf3b9075"
server_key_iv: "97431d0b35af55f1dbd82e54e6b5232c"
server_key_version: "encryptionkey@0"
I20260810 06:37:28.814805 2723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:28.828251 2723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:28.832015 2723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:28.832264 2723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:28.834748 2723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:28.834848 2723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:28.834956 2723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:28.835072 2723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:37:28 dist-test-slave-4tdj krb5kdc[2495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343848, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:28.857285 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:28.835271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:38821 (local address 127.31.59.254:38471)
0810 06:37:28.835909 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:28.835915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:28.836871 (+ 956us) server_negotiation.cc:407] Connection header received
0810 06:37:28.839105 (+ 2234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:28.839112 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:28.839205 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:28.839326 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:28.841412 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:28.842435 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:28.844830 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:28.845493 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:28.845664 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:28.850411 (+ 4747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:28.850418 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:28.850423 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:28.850465 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:28.853361 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:28.854400 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:28.854405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:28.854408 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:28.854582 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:28.855192 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:28.855197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:28.855201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:28.855474 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:28.855591 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:28.856651 (+ 1060us) server_negotiation.cc:299] Negotiation successful
0810 06:37:28.856884 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":418,"threads_started":1}
I20260810 06:37:28.858965 2763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:28.835876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:38471 (local address 127.31.59.193:38821)
0810 06:37:28.836696 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:28.836751 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:28.838754 (+ 2003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:28.839618 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:28.839633 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:28.840349 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:28.841176 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:28.841205 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:28.842586 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:28.842591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:28.844672 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:28.844684 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:28.845746 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:28.845753 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:28.845931 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:28.846626 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:28.846658 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:28.850129 (+ 3471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:28.853503 (+ 3374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:28.853514 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:28.853531 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:28.854258 (+ 727us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:28.854717 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:28.854722 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:28.854727 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:28.855049 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:28.855608 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:28.855616 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:28.856161 (+ 545us) client_negotiation.cc:769] Sending connection context
0810 06:37:28.856512 (+ 351us) client_negotiation.cc:240] Negotiation successful
0810 06:37:28.857132 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":344,"threads_started":1}
I20260810 06:37:28.877101 2723 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:35223
I20260810 06:37:28.877179 2964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:35223 every 8 connection(s)
I20260810 06:37:28.878460 2723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:37:28.880959 2966 heartbeater.cc:347] Connected to a master server at 127.31.59.254:38471
I20260810 06:37:28.881229 2966 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:28.881620 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 2723
I20260810 06:37:28.881834 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:37:28.882162 2966 heartbeater.cc:511] Master 127.31.59.254:38471 requested a full tablet report, sending...
I20260810 06:37:28.882512 31983 external_mini_cluster.cc:1428] Setting key b1898e7f39daa5a3d7cc852f9511ba5f
I20260810 06:37:28.882794 2929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:28.867317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:38471 (local address 127.31.59.193:50807)
0810 06:37:28.868086 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:28.868107 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:28.868236 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:28.868611 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:28.868616 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:28.869045 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:28.869345 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:28.869355 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:28.870760 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:28.870771 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:28.872506 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:28.872524 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:28.874615 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:28.874620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:28.874742 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:28.875424 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:28.875445 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:28.876196 (+ 751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:28.879725 (+ 3529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:28.879731 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:28.879735 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:28.880180 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:28.880666 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:28.880672 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:28.880676 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:28.880808 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:28.881489 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:28.881495 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:28.881720 (+ 225us) client_negotiation.cc:769] Sending connection context
0810 06:37:28.881878 (+ 158us) client_negotiation.cc:240] Negotiation successful
0810 06:37:28.882240 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":3,"thread_start_us":372,"threads_started":1}
I20260810 06:37:28.883894 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:28.867194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:50807 (local address 127.31.59.254:38471)
0810 06:37:28.867371 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:28.867377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:28.868192 (+ 815us) server_negotiation.cc:407] Connection header received
0810 06:37:28.868366 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:28.868371 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:28.868454 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:28.868553 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:28.869476 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:28.870593 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:28.873748 (+ 3155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:28.874438 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:28.874563 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:28.876386 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:28.876392 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:28.876396 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:28.876432 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:28.879552 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:28.880349 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:28.880355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:28.880359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:28.880536 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:28.880917 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:28.880922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:28.880925 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:28.881318 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:28.883177 (+ 1859us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:28.883292 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 06:37:28.883525 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:37:28.884681 2575 ts_manager.cc:194] Registered new tserver with Master: e4125072468246afa375da1a22c5be30 (127.31.59.193:35223)
I20260810 06:37:28.886685 2575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:38821
WARNING: no policy specified for kudu/127.31.59.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
I20260810 06:37:28.923362 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:38471
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:35445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:29.236631 2973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:29.236891 2973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:29.236990 2973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:29.244014 2973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:29.244115 2973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:29.244205 2973 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 06:37:29.244259 2973 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:37:29.244318 2973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:37:29.253736 2973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:35445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.59.254:38471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:29.255059 2973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:29.256332 2973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:29.268851 2985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:29.268828 2986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:29.269371 2988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:29.269840 2973 server_base.cc:1034] running on GCE node
I20260810 06:37:29.270499 2973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:29.271412 2973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:29.272637 2973 hybrid_clock.cc:648] HybridClock initialized: now 1786343849272584 us; error 43 us; skew 500 ppm
I20260810 06:37:29.272936 2973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:29 dist-test-slave-4tdj krb5kdc[2495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343849, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:29.275997 2973 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:37:29.277346 2973 webserver.cc:460] Webserver started at http://127.31.59.194:46313/ using document root <none> and password file <none>
I20260810 06:37:29.277999 2973 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:29.278118 2973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:29.278452 2973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:29.281245 2973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "c083e39d169f46c7b8e79b96aa62e3dd"
format_stamp: "Formatted at 2026-08-10 06:37:29 on dist-test-slave-4tdj"
server_key: "5ddb6100cc5deb0da7942020f321e6fa"
server_key_iv: "8fe59d00e18cf69943bd7723e647d313"
server_key_version: "encryptionkey@0"
I20260810 06:37:29.281961 2973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "c083e39d169f46c7b8e79b96aa62e3dd"
format_stamp: "Formatted at 2026-08-10 06:37:29 on dist-test-slave-4tdj"
server_key: "5ddb6100cc5deb0da7942020f321e6fa"
server_key_iv: "8fe59d00e18cf69943bd7723e647d313"
server_key_version: "encryptionkey@0"
I20260810 06:37:29.286875 2973 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260810 06:37:29.290096 2997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:29.291363 2973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:37:29.291504 2973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "c083e39d169f46c7b8e79b96aa62e3dd"
format_stamp: "Formatted at 2026-08-10 06:37:29 on dist-test-slave-4tdj"
server_key: "5ddb6100cc5deb0da7942020f321e6fa"
server_key_iv: "8fe59d00e18cf69943bd7723e647d313"
server_key_version: "encryptionkey@0"
I20260810 06:37:29.291702 2973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:29.307425 2973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:29.311343 2973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:29.311604 2973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:29.314201 2973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:29.314322 2973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:29.314435 2973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:29.314535 2973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:37:29 dist-test-slave-4tdj krb5kdc[2495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343849, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:29.336863 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:29.314741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:46137 (local address 127.31.59.254:38471)
0810 06:37:29.314940 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:29.314946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:29.316270 (+ 1324us) server_negotiation.cc:407] Connection header received
0810 06:37:29.318434 (+ 2164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:29.318440 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:29.318516 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:29.318604 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:29.320709 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.321883 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.324311 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.324820 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.324943 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:29.329711 (+ 4768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:29.329718 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:29.329724 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:29.329768 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:29.332951 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:29.333994 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:29.334000 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:29.334002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:29.334147 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:29.334718 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:29.334723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:29.334727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:29.335109 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:29.335247 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:29.336200 (+ 953us) server_negotiation.cc:299] Negotiation successful
0810 06:37:29.336452 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 06:37:29.337607 3012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:29.315260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:38471 (local address 127.31.59.194:46137)
0810 06:37:29.316083 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:29.316141 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:29.318113 (+ 1972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:29.319016 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:29.319027 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:29.319714 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:29.320505 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:29.320534 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.322048 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.322054 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:29.324154 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:29.324169 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.325086 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.325094 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:29.325396 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:29.326202 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:29.326232 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:29.329449 (+ 3217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:29.333118 (+ 3669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:29.333128 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:29.333147 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:29.333849 (+ 702us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:29.334266 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:29.334271 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:29.334275 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:29.334601 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:29.335224 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:29.335235 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:29.335722 (+ 487us) client_negotiation.cc:769] Sending connection context
0810 06:37:29.336066 (+ 344us) client_negotiation.cc:240] Negotiation successful
0810 06:37:29.336459 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":349,"threads_started":1}
I20260810 06:37:29.357897 3169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:29.344145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:38471 (local address 127.31.59.194:57585)
0810 06:37:29.344870 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:29.344892 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:29.345040 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:29.345344 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:29.345349 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:29.345995 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:29.346408 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:29.346440 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.347518 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.347523 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:29.349163 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:29.349180 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.349938 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.349944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:29.350125 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:29.350840 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:29.350866 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:29.351978 (+ 1112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:29.355325 (+ 3347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:29.355331 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:29.355335 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:29.355878 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:29.356296 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:29.356299 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:29.356302 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:29.356414 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:29.357067 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:29.357070 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:29.357223 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 06:37:29.357345 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 06:37:29.357569 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":576,"threads_started":1}
I20260810 06:37:29.358033 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:29.344670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:57585 (local address 127.31.59.254:38471)
0810 06:37:29.344818 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:29.344824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:29.344978 (+ 154us) server_negotiation.cc:407] Connection header received
0810 06:37:29.345152 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:29.345156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:29.345225 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:29.345300 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:29.346595 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.347403 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.349323 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.349769 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.349869 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:29.352165 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:29.352170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:29.352174 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:29.352207 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:29.355168 (+ 2961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:29.356011 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:29.356017 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:29.356021 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:29.356194 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:29.356494 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:29.356507 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:29.356510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:29.356962 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:29.357115 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:29.357453 (+ 338us) server_negotiation.cc:299] Negotiation successful
0810 06:37:29.357700 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 06:37:29.362208 2973 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:39235
I20260810 06:37:29.362330 3210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:39235 every 8 connection(s)
I20260810 06:37:29.363440 2973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:37:29.365298 3211 heartbeater.cc:347] Connected to a master server at 127.31.59.254:38471
I20260810 06:37:29.365505 3211 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:29.366118 3211 heartbeater.cc:511] Master 127.31.59.254:38471 requested a full tablet report, sending...
I20260810 06:37:29.367367 2575 ts_manager.cc:194] Registered new tserver with Master: c083e39d169f46c7b8e79b96aa62e3dd (127.31.59.194:39235)
I20260810 06:37:29.368206 2575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:46137
I20260810 06:37:29.368863 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 2973
I20260810 06:37:29.369041 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:37:29.369426 31983 external_mini_cluster.cc:1428] Setting key 77f14b2ae677c1278dbe0a0ad90bccd0
WARNING: no policy specified for kudu/127.31.59.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
I20260810 06:37:29.409369 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:38471
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:35445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:29.719440 3215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:29.719727 3215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:29.719877 3215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:29.726917 3215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:29.727032 3215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:29.727174 3215 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 06:37:29.727253 3215 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:37:29.727358 3215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:37:29.736773 3215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:35445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.59.254:38471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:29.738008 3215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:29.739351 3215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:29.752290 3223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:29.752357 3228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:29.752388 3225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:29.753307 3215 server_base.cc:1034] running on GCE node
I20260810 06:37:29.753866 3215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:29.754725 3215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:29.755927 3215 hybrid_clock.cc:648] HybridClock initialized: now 1786343849755879 us; error 39 us; skew 500 ppm
I20260810 06:37:29.756215 3215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:29 dist-test-slave-4tdj krb5kdc[2495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343849, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:29.759459 3215 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:37:29.760885 3215 webserver.cc:460] Webserver started at http://127.31.59.195:41139/ using document root <none> and password file <none>
I20260810 06:37:29.761529 3215 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:29.761644 3215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:29.761942 3215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:29.764727 3215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "62cc8fd91a0544cebe5825c9d9651c6e"
format_stamp: "Formatted at 2026-08-10 06:37:29 on dist-test-slave-4tdj"
server_key: "b70b2b99b05cedbc5572eeeccafbe9c5"
server_key_iv: "7d309767a401245c4930d3ed6094cd9f"
server_key_version: "encryptionkey@0"
I20260810 06:37:29.765427 3215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "62cc8fd91a0544cebe5825c9d9651c6e"
format_stamp: "Formatted at 2026-08-10 06:37:29 on dist-test-slave-4tdj"
server_key: "b70b2b99b05cedbc5572eeeccafbe9c5"
server_key_iv: "7d309767a401245c4930d3ed6094cd9f"
server_key_version: "encryptionkey@0"
I20260810 06:37:29.770551 3215 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 06:37:29.773855 3239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:29.775182 3215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:37:29.775314 3215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "62cc8fd91a0544cebe5825c9d9651c6e"
format_stamp: "Formatted at 2026-08-10 06:37:29 on dist-test-slave-4tdj"
server_key: "b70b2b99b05cedbc5572eeeccafbe9c5"
server_key_iv: "7d309767a401245c4930d3ed6094cd9f"
server_key_version: "encryptionkey@0"
I20260810 06:37:29.775471 3215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:29.793821 3215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:29.797811 3215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:29.798048 3215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:29.800762 3215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:29.800853 3215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:29.800963 3215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:29.801061 3215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:37:29 dist-test-slave-4tdj krb5kdc[2495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343849, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:29.823060 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:29.801285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:44053 (local address 127.31.59.254:38471)
0810 06:37:29.801507 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:29.801513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:29.803001 (+ 1488us) server_negotiation.cc:407] Connection header received
0810 06:37:29.805192 (+ 2191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:29.805198 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:29.805305 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:29.805403 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:29.807669 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.808814 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.810501 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.811118 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.811256 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:29.816461 (+ 5205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:29.816469 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:29.816474 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:29.816523 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:29.819710 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:29.820592 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:29.820596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:29.820598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:29.820744 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:29.821273 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:29.821277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:29.821280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:29.821559 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:29.821676 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:29.822609 (+ 933us) server_negotiation.cc:299] Negotiation successful
0810 06:37:29.822785 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 06:37:29.824826 3250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:29.801874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:38471 (local address 127.31.59.195:44053)
0810 06:37:29.802803 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:29.802856 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:29.804852 (+ 1996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:29.805714 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:29.805727 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:29.806593 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:29.807449 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:29.807483 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.809006 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.809012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:29.810336 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:29.810354 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.811594 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.811601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:29.811853 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:29.812595 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:29.812633 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:29.816170 (+ 3537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:29.819850 (+ 3680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:29.819860 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:29.819878 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:29.820470 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:29.820841 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:29.820846 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:29.820851 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:29.821121 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:29.821671 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:29.821679 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:29.822141 (+ 462us) client_negotiation.cc:769] Sending connection context
0810 06:37:29.822464 (+ 323us) client_negotiation.cc:240] Negotiation successful
0810 06:37:29.823115 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":404,"threads_started":1}
I20260810 06:37:29.847502 3215 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:44383
I20260810 06:37:29.847551 3463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:44383 every 8 connection(s)
I20260810 06:37:29.848897 3215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:37:29.850965 3464 heartbeater.cc:347] Connected to a master server at 127.31.59.254:38471
I20260810 06:37:29.851186 3464 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:29.851858 3464 heartbeater.cc:511] Master 127.31.59.254:38471 requested a full tablet report, sending...
I20260810 06:37:29.852572 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:29.838285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:42539 (local address 127.31.59.254:38471)
0810 06:37:29.838505 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:29.838512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:29.839245 (+ 733us) server_negotiation.cc:407] Connection header received
0810 06:37:29.839426 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:29.839430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:29.839490 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:29.839568 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:29.841002 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.841902 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.844160 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.844687 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.844814 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:29.847195 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:29.847199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:29.847202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:29.847237 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:29.849819 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:29.850618 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:29.850624 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:29.850628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:29.850758 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:29.851101 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:29.851106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:29.851110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:29.851424 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:29.851576 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:29.852009 (+ 433us) server_negotiation.cc:299] Negotiation successful
0810 06:37:29.852224 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:37:29.852632 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:29.838368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:38471 (local address 127.31.59.195:42539)
0810 06:37:29.839130 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:29.839160 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:29.839319 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:29.839630 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:29.839635 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:29.840242 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:29.840784 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:29.840799 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.842059 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.842084 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:29.843999 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:29.844016 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.845034 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.845044 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:29.845234 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:29.845974 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:29.845996 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:29.846999 (+ 1003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:29.849970 (+ 2971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:29.849979 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:29.849982 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:29.850500 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:29.850862 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:29.850867 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:29.850871 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:29.850994 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:29.851559 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:29.851563 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:29.851744 (+ 181us) client_negotiation.cc:769] Sending connection context
0810 06:37:29.851862 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 06:37:29.852165 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":449,"threads_started":1}
I20260810 06:37:29.853243 2575 ts_manager.cc:194] Registered new tserver with Master: 62cc8fd91a0544cebe5825c9d9651c6e (127.31.59.195:44383)
I20260810 06:37:29.854264 2575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:44053
I20260810 06:37:29.856002 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 3215
I20260810 06:37:29.856200 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:37:29.856752 31983 external_mini_cluster.cc:1428] Setting key 9d2101b39a76c7967f58c4c6e0d1c3ef
I20260810 06:37:29.859416 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 06:37:29 dist-test-slave-4tdj krb5kdc[2495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260810 06:37:29.889758 2966 heartbeater.cc:503] Master 127.31.59.254:38471 was elected leader, sending a full tablet report...
Aug 10 06:37:29 dist-test-slave-4tdj krb5kdc[2495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:29.906605 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:29.893551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42860 (local address 127.31.59.254:38471)
0810 06:37:29.893700 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:29.893704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:29.894129 (+ 425us) server_negotiation.cc:407] Connection header received
0810 06:37:29.894276 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:29.894280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:29.894388 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:29.894472 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:29.895912 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.896707 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.897845 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.898243 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.898381 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:29.901879 (+ 3498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:29.901884 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:29.901888 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:29.901920 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:29.904313 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:29.904934 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:29.904938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:29.904940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:29.905076 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:29.905375 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:29.905378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:29.905380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:29.905666 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:29.905844 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:29.906035 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 06:37:29.906209 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:37:29.910372 2575 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:37:29.917001 2575 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:37:29.944684 3485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:29.929456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43202 (local address 127.31.59.195:44383)
0810 06:37:29.934650 (+ 5194us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:29.934660 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:29.935721 (+ 1061us) server_negotiation.cc:407] Connection header received
0810 06:37:29.936020 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:29.936029 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:29.936254 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:29.936406 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:29.938428 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.939951 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.942130 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.943468 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.943597 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:29.943854 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:29.944155 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 06:37:29.944324 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5008,"thread_start_us":4835,"threads_started":1}
I20260810 06:37:29.944756 3481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:29.929196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:44383 (local address 127.0.0.1:43202)
0810 06:37:29.935116 (+ 5920us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:29.935629 (+ 513us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:29.935828 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:29.936824 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:29.936832 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:29.936885 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:29.937403 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:29.937417 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.940119 (+ 2702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.940125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:29.941932 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:29.941950 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.943712 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.943719 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:29.943918 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:29.943970 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 06:37:29.944099 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 06:37:29.944312 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5617,"mutex_wait_us":1,"spinlock_wait_cycles":4224,"thread_start_us":5444,"threads_started":1}
I20260810 06:37:29.944916 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:29.929820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48122 (local address 127.31.59.194:39235)
0810 06:37:29.934060 (+ 4240us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:29.934072 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:29.935328 (+ 1256us) server_negotiation.cc:407] Connection header received
0810 06:37:29.935767 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:29.935777 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:29.936060 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:29.936255 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:29.937818 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.939361 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.942194 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.943555 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.943687 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:29.943865 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:29.944455 (+ 590us) server_negotiation.cc:299] Negotiation successful
0810 06:37:29.944624 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4021,"thread_start_us":4228,"threads_started":1}
I20260810 06:37:29.945727 3483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:29.929395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:39235 (local address 127.0.0.1:48122)
0810 06:37:29.935116 (+ 5721us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:29.935188 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:29.935489 (+ 301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:29.936824 (+ 1335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:29.936832 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:29.936876 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:29.937406 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:29.937417 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.939582 (+ 2165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.939589 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:29.942052 (+ 2463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:29.942069 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.944031 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.944038 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:29.944255 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:29.944287 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:37:29.945191 (+ 904us) client_negotiation.cc:240] Negotiation successful
0810 06:37:29.945336 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5597,"mutex_wait_us":75,"spinlock_wait_cycles":1408,"thread_start_us":5743,"threads_started":1}
I20260810 06:37:29.946982 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:29.935073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:35223 (local address 127.0.0.1:52006)
0810 06:37:29.938428 (+ 3355us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:29.938448 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:29.938582 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:29.939220 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:29.939225 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:29.939251 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:29.939691 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:29.939704 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.941581 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.941586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:29.944077 (+ 2491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:29.944091 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.946205 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.946211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:29.946398 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:29.946429 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:37:29.946528 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 06:37:29.946642 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3199,"mutex_wait_us":6,"thread_start_us":3047,"threads_started":1}
I20260810 06:37:29.947438 3490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:29.935282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52006 (local address 127.31.59.193:35223)
0810 06:37:29.937976 (+ 2694us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:29.937985 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:29.938710 (+ 725us) server_negotiation.cc:407] Connection header received
0810 06:37:29.938821 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:29.938829 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:29.939030 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:29.939221 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:29.939898 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.941448 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.944255 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:29.945741 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:29.945881 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:29.946068 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:29.947004 (+ 936us) server_negotiation.cc:299] Negotiation successful
0810 06:37:29.947166 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2537,"thread_start_us":2377,"threads_started":1}
I20260810 06:37:29.949724 3076 tablet_service.cc:1467] Processing CreateTablet for tablet 75820b092dca4b0685bfa064fe46954f (DEFAULT_TABLE table=test-table [id=339706564fef4ca5bdf755fde001d849]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:29.949729 3336 tablet_service.cc:1467] Processing CreateTablet for tablet 75820b092dca4b0685bfa064fe46954f (DEFAULT_TABLE table=test-table [id=339706564fef4ca5bdf755fde001d849]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:29.950901 2843 tablet_service.cc:1467] Processing CreateTablet for tablet 75820b092dca4b0685bfa064fe46954f (DEFAULT_TABLE table=test-table [id=339706564fef4ca5bdf755fde001d849]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:29.951334 3336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75820b092dca4b0685bfa064fe46954f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:29.951335 3076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75820b092dca4b0685bfa064fe46954f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:29.952041 2843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75820b092dca4b0685bfa064fe46954f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:29.963974 3491 tablet_bootstrap.cc:492] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30: Bootstrap starting.
I20260810 06:37:29.967752 3492 tablet_bootstrap.cc:492] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e: Bootstrap starting.
I20260810 06:37:29.967964 3491 tablet_bootstrap.cc:654] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:29.968264 3494 tablet_bootstrap.cc:492] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd: Bootstrap starting.
I20260810 06:37:29.969389 3491 log.cc:824] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:29.971982 3494 tablet_bootstrap.cc:654] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:29.972175 3491 tablet_bootstrap.cc:492] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30: No bootstrap required, opened a new log
I20260810 06:37:29.972429 3491 ts_tablet_manager.cc:1397] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 06:37:29.973208 3494 log.cc:824] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:29.973500 3492 tablet_bootstrap.cc:654] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:29.974565 3492 log.cc:824] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:29.976085 3494 tablet_bootstrap.cc:492] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd: No bootstrap required, opened a new log
I20260810 06:37:29.976351 3494 ts_tablet_manager.cc:1397] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260810 06:37:29.977433 3492 tablet_bootstrap.cc:492] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e: No bootstrap required, opened a new log
I20260810 06:37:29.977697 3492 ts_tablet_manager.cc:1397] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260810 06:37:29.980144 3491 raft_consensus.cc:348] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } }
I20260810 06:37:29.980573 3491 raft_consensus.cc:374] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:29.980676 3491 raft_consensus.cc:729] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4125072468246afa375da1a22c5be30, State: Initialized, Role: FOLLOWER
I20260810 06:37:29.981262 3491 consensus_queue.cc:260] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } }
I20260810 06:37:29.982982 3491 ts_tablet_manager.cc:1428] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30: Time spent starting tablet: real 0.010s user 0.012s sys 0.000s
I20260810 06:37:29.986579 3494 raft_consensus.cc:348] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } }
I20260810 06:37:29.986990 3494 raft_consensus.cc:374] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:29.987092 3494 raft_consensus.cc:729] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c083e39d169f46c7b8e79b96aa62e3dd, State: Initialized, Role: FOLLOWER
I20260810 06:37:29.987679 3494 consensus_queue.cc:260] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } }
I20260810 06:37:29.987728 3492 raft_consensus.cc:348] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } }
I20260810 06:37:29.988109 3492 raft_consensus.cc:374] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:29.988219 3492 raft_consensus.cc:729] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62cc8fd91a0544cebe5825c9d9651c6e, State: Initialized, Role: FOLLOWER
I20260810 06:37:29.988896 3211 heartbeater.cc:503] Master 127.31.59.254:38471 was elected leader, sending a full tablet report...
I20260810 06:37:29.988767 3492 consensus_queue.cc:260] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } }
I20260810 06:37:29.989339 3494 ts_tablet_manager.cc:1428] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd: Time spent starting tablet: real 0.013s user 0.004s sys 0.010s
I20260810 06:37:29.989816 3464 heartbeater.cc:503] Master 127.31.59.254:38471 was elected leader, sending a full tablet report...
I20260810 06:37:29.990459 3492 ts_tablet_manager.cc:1428] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e: Time spent starting tablet: real 0.013s user 0.013s sys 0.002s
W20260810 06:37:30.100613 3465 tablet.cc:2369] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:37:30.115312 3212 tablet.cc:2369] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:37:30.130860 2967 tablet.cc:2369] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:37:30.145491 3502 raft_consensus.cc:482] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:37:30.145814 3502 raft_consensus.cc:504] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } }
I20260810 06:37:30.147778 3502 leader_election.cc:288] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c083e39d169f46c7b8e79b96aa62e3dd (127.31.59.194:39235), e4125072468246afa375da1a22c5be30 (127.31.59.193:35223)
I20260810 06:37:30.153730 3250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:30.148066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:35223 (local address 127.31.59.195:41677)
0810 06:37:30.148366 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:30.148382 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:30.148504 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:30.148861 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:30.148865 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:30.148905 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:30.149122 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:30.149136 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.150350 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.150355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:30.151865 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:30.151881 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.153185 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.153190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:30.153331 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:30.153356 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 06:37:30.153426 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 06:37:30.153523 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260810 06:37:30.153836 3490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:30.148373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:41677 (local address 127.31.59.193:35223)
0810 06:37:30.148609 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:30.148615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:30.148634 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:37:30.148692 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:30.148695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:30.148748 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:30.148822 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:30.149268 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.150193 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.152051 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.153054 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.153209 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:30.153284 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:30.153465 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 06:37:30.153606 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:37:30.154590 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:30.148109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:34343 (local address 127.31.59.194:39235)
0810 06:37:30.148309 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:30.148316 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:30.149016 (+ 700us) server_negotiation.cc:407] Connection header received
0810 06:37:30.149168 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:30.149173 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:30.149253 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:30.149322 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:30.149899 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.150870 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.152518 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.153627 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.153789 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:30.153858 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:30.154233 (+ 375us) server_negotiation.cc:299] Negotiation successful
0810 06:37:30.154394 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:37:30.154645 3503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:30.148187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:39235 (local address 127.31.59.195:34343)
0810 06:37:30.148916 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:30.148930 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:30.149052 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:30.149359 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:30.149364 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:30.149388 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:30.149728 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:30.149739 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.151005 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.151009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:30.152367 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:30.152379 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.153852 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.153858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:30.154007 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:30.154044 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 06:37:30.154180 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 06:37:30.154314 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":405,"threads_started":1}
I20260810 06:37:30.155161 2887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75820b092dca4b0685bfa064fe46954f" candidate_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4125072468246afa375da1a22c5be30" is_pre_election: true
I20260810 06:37:30.155534 3116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75820b092dca4b0685bfa064fe46954f" candidate_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" is_pre_election: true
I20260810 06:37:30.155663 2887 raft_consensus.cc:2454] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62cc8fd91a0544cebe5825c9d9651c6e in term 0.
I20260810 06:37:30.155963 3116 raft_consensus.cc:2454] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62cc8fd91a0544cebe5825c9d9651c6e in term 0.
I20260810 06:37:30.156436 3244 leader_election.cc:302] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62cc8fd91a0544cebe5825c9d9651c6e, e4125072468246afa375da1a22c5be30; no voters:
I20260810 06:37:30.156823 3502 raft_consensus.cc:2781] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:37:30.156951 3502 raft_consensus.cc:482] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:37:30.157052 3502 raft_consensus.cc:3037] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:30.159226 3502 raft_consensus.cc:504] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } }
I20260810 06:37:30.159837 3502 leader_election.cc:288] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [CANDIDATE]: Term 1 election: Requested vote from peers c083e39d169f46c7b8e79b96aa62e3dd (127.31.59.194:39235), e4125072468246afa375da1a22c5be30 (127.31.59.193:35223)
I20260810 06:37:30.160247 3116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75820b092dca4b0685bfa064fe46954f" candidate_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c083e39d169f46c7b8e79b96aa62e3dd"
I20260810 06:37:30.160324 2887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75820b092dca4b0685bfa064fe46954f" candidate_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4125072468246afa375da1a22c5be30"
I20260810 06:37:30.160415 3116 raft_consensus.cc:3037] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:30.160511 2887 raft_consensus.cc:3037] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:30.162652 3116 raft_consensus.cc:2454] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62cc8fd91a0544cebe5825c9d9651c6e in term 1.
I20260810 06:37:30.163009 3241 leader_election.cc:302] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62cc8fd91a0544cebe5825c9d9651c6e, c083e39d169f46c7b8e79b96aa62e3dd; no voters:
I20260810 06:37:30.163064 2887 raft_consensus.cc:2454] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62cc8fd91a0544cebe5825c9d9651c6e in term 1.
I20260810 06:37:30.163269 3502 raft_consensus.cc:2781] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:30.163693 3502 raft_consensus.cc:686] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 LEADER]: Becoming Leader. State: Replica: 62cc8fd91a0544cebe5825c9d9651c6e, State: Running, Role: LEADER
I20260810 06:37:30.164120 3502 consensus_queue.cc:237] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } }
I20260810 06:37:30.168682 2576 catalog_manager.cc:5495] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 62cc8fd91a0544cebe5825c9d9651c6e (127.31.59.195). New cstate: current_term: 1 leader_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } health_report { overall_health: UNKNOWN } } }
Aug 10 06:37:30 dist-test-slave-4tdj krb5kdc[2495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343848, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:30.269351 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:30.256785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:38471 (local address 127.0.0.1:42874)
0810 06:37:30.257325 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:30.257341 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:30.257468 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:30.257786 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:30.257789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:30.258496 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:30.258757 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:30.258766 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.259717 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.259721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:30.260678 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:30.260689 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.261393 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.261398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:30.261548 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:30.262151 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:30.262169 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:30.264314 (+ 2145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:30.266959 (+ 2645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:30.266969 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:30.266974 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:30.267430 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:30.267795 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:30.267799 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:30.267801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:30.267955 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:30.268485 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:30.268491 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:30.268673 (+ 182us) client_negotiation.cc:769] Sending connection context
0810 06:37:30.268813 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 06:37:30.269009 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":284,"threads_started":1}
I20260810 06:37:30.269357 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:30.256739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42874 (local address 127.31.59.254:38471)
0810 06:37:30.256913 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:30.256917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:30.257421 (+ 504us) server_negotiation.cc:407] Connection header received
0810 06:37:30.257586 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:30.257590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:30.257662 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:30.257742 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:30.258886 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.259610 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.260795 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.261228 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.261312 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:30.264479 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:30.264484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:30.264488 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:30.264525 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:30.266838 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:30.267537 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:30.267541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:30.267543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:30.267658 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:30.268078 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:30.268081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:30.268083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:30.268371 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:30.268474 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:30.268914 (+ 440us) server_negotiation.cc:299] Negotiation successful
0810 06:37:30.269116 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":62}
I20260810 06:37:30.274113 2576 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:42874:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 06:37:30.275187 2576 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:37:30.281641 3076 tablet_service.cc:1467] Processing CreateTablet for tablet 4db18e6a17ce42c6bf93532bd09c7111 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=713d67439a28450983ced2bca7dd4ee6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:37:30.281545 2843 tablet_service.cc:1467] Processing CreateTablet for tablet 4db18e6a17ce42c6bf93532bd09c7111 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=713d67439a28450983ced2bca7dd4ee6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:37:30.281944 3336 tablet_service.cc:1467] Processing CreateTablet for tablet 4db18e6a17ce42c6bf93532bd09c7111 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=713d67439a28450983ced2bca7dd4ee6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:37:30.282563 3076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db18e6a17ce42c6bf93532bd09c7111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:30.282562 2843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db18e6a17ce42c6bf93532bd09c7111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:30.282742 3336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db18e6a17ce42c6bf93532bd09c7111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:30.288074 3494 tablet_bootstrap.cc:492] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd: Bootstrap starting.
I20260810 06:37:30.289373 3494 tablet_bootstrap.cc:654] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:30.289496 3492 tablet_bootstrap.cc:492] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e: Bootstrap starting.
I20260810 06:37:30.289896 3491 tablet_bootstrap.cc:492] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30: Bootstrap starting.
I20260810 06:37:30.291132 3492 tablet_bootstrap.cc:654] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:30.291384 3491 tablet_bootstrap.cc:654] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:30.292021 3494 tablet_bootstrap.cc:492] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd: No bootstrap required, opened a new log
I20260810 06:37:30.292179 3494 ts_tablet_manager.cc:1397] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260810 06:37:30.292934 3494 raft_consensus.cc:348] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:30.293164 3494 raft_consensus.cc:374] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:30.293190 3492 tablet_bootstrap.cc:492] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e: No bootstrap required, opened a new log
I20260810 06:37:30.293252 3494 raft_consensus.cc:729] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c083e39d169f46c7b8e79b96aa62e3dd, State: Initialized, Role: FOLLOWER
I20260810 06:37:30.293318 3492 ts_tablet_manager.cc:1397] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260810 06:37:30.293481 3494 consensus_queue.cc:260] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:30.293617 3491 tablet_bootstrap.cc:492] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30: No bootstrap required, opened a new log
I20260810 06:37:30.293741 3491 ts_tablet_manager.cc:1397] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 06:37:30.294025 3494 ts_tablet_manager.cc:1428] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 06:37:30.293916 3499 tablet_replica.cc:439] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } }
I20260810 06:37:30.294093 3492 raft_consensus.cc:348] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:30.294312 3499 tablet_replica.cc:442] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:37:30.294340 3492 raft_consensus.cc:374] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:30.294435 3492 raft_consensus.cc:729] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62cc8fd91a0544cebe5825c9d9651c6e, State: Initialized, Role: FOLLOWER
I20260810 06:37:30.294639 3492 consensus_queue.cc:260] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:30.294682 3491 raft_consensus.cc:348] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:30.294945 3491 raft_consensus.cc:374] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:30.295034 3491 raft_consensus.cc:729] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4125072468246afa375da1a22c5be30, State: Initialized, Role: FOLLOWER
I20260810 06:37:30.295073 3502 tablet_replica.cc:439] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } }
I20260810 06:37:30.295365 3502 tablet_replica.cc:442] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:37:30.295258 3491 consensus_queue.cc:260] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:30.295219 3492 ts_tablet_manager.cc:1428] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:37:30.295879 3491 ts_tablet_manager.cc:1428] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:37:30.295734 3498 tablet_replica.cc:439] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } }
I20260810 06:37:30.296154 3498 tablet_replica.cc:442] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:37:30.357774 3502 raft_consensus.cc:482] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:37:30.358011 3502 raft_consensus.cc:504] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:30.358705 3502 leader_election.cc:288] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4125072468246afa375da1a22c5be30 (127.31.59.193:35223), c083e39d169f46c7b8e79b96aa62e3dd (127.31.59.194:39235)
I20260810 06:37:30.359191 2887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db18e6a17ce42c6bf93532bd09c7111" candidate_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4125072468246afa375da1a22c5be30" is_pre_election: true
I20260810 06:37:30.359280 3116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db18e6a17ce42c6bf93532bd09c7111" candidate_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" is_pre_election: true
I20260810 06:37:30.359434 2887 raft_consensus.cc:2454] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62cc8fd91a0544cebe5825c9d9651c6e in term 0.
I20260810 06:37:30.359598 3116 raft_consensus.cc:2454] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62cc8fd91a0544cebe5825c9d9651c6e in term 0.
I20260810 06:37:30.359839 3244 leader_election.cc:302] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62cc8fd91a0544cebe5825c9d9651c6e, e4125072468246afa375da1a22c5be30; no voters:
I20260810 06:37:30.360142 3502 raft_consensus.cc:2781] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:37:30.360231 3502 raft_consensus.cc:482] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:37:30.360334 3502 raft_consensus.cc:3037] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:30.362732 3502 raft_consensus.cc:504] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:30.363230 3502 leader_election.cc:288] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [CANDIDATE]: Term 1 election: Requested vote from peers e4125072468246afa375da1a22c5be30 (127.31.59.193:35223), c083e39d169f46c7b8e79b96aa62e3dd (127.31.59.194:39235)
I20260810 06:37:30.363710 2887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db18e6a17ce42c6bf93532bd09c7111" candidate_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4125072468246afa375da1a22c5be30"
I20260810 06:37:30.363893 2887 raft_consensus.cc:3037] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:30.365796 2887 raft_consensus.cc:2454] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62cc8fd91a0544cebe5825c9d9651c6e in term 1.
I20260810 06:37:30.366197 3244 leader_election.cc:302] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62cc8fd91a0544cebe5825c9d9651c6e, e4125072468246afa375da1a22c5be30; no voters:
I20260810 06:37:30.366137 3116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db18e6a17ce42c6bf93532bd09c7111" candidate_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c083e39d169f46c7b8e79b96aa62e3dd"
I20260810 06:37:30.366410 3116 raft_consensus.cc:3037] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:30.366460 3502 raft_consensus.cc:2781] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:30.366640 3502 raft_consensus.cc:686] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 LEADER]: Becoming Leader. State: Replica: 62cc8fd91a0544cebe5825c9d9651c6e, State: Running, Role: LEADER
I20260810 06:37:30.366878 3502 consensus_queue.cc:237] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:30.368352 3504 tablet_replica.cc:439] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e: TxnStatusTablet state changed. Reason: New leader 62cc8fd91a0544cebe5825c9d9651c6e. Latest consensus state: current_term: 1 leader_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } }
I20260810 06:37:30.368646 3504 tablet_replica.cc:442] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:37:30.369560 3116 raft_consensus.cc:2454] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62cc8fd91a0544cebe5825c9d9651c6e in term 1.
I20260810 06:37:30.371565 2576 catalog_manager.cc:5495] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 62cc8fd91a0544cebe5825c9d9651c6e (127.31.59.195). New cstate: current_term: 1 leader_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } health_report { overall_health: HEALTHY } } }
I20260810 06:37:30.377266 3531 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 06:37:30.377543 3531 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 06:37:30.394935 3485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:30.389520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43210 (local address 127.31.59.195:44383)
0810 06:37:30.389700 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:30.389706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:30.389846 (+ 140us) server_negotiation.cc:407] Connection header received
0810 06:37:30.390127 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:30.390130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:30.390214 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:30.390386 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:37:30.390728 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.391560 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.392799 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.393236 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.393341 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:30.393805 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:37:30.394257 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:37:30.394451 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:30.394569 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 06:37:30.394652 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:37:30.394960 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:30.389427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:44383 (local address 127.0.0.1:43210)
0810 06:37:30.389756 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:30.389772 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:30.389909 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:30.390405 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:30.390410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:30.390422 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:37:30.390606 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:30.390615 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.391683 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.391688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:30.392664 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:30.392676 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.393384 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.393390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:30.393566 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:30.393617 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:37:30.394422 (+ 805us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:37:30.394443 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 06:37:30.394529 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 06:37:30.394676 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260810 06:37:30.403532 3116 raft_consensus.cc:1261] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Refusing update from remote peer 62cc8fd91a0544cebe5825c9d9651c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:30.404177 2887 raft_consensus.cc:1261] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30 [term 1 FOLLOWER]: Refusing update from remote peer 62cc8fd91a0544cebe5825c9d9651c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:30.404353 3504 consensus_queue.cc:1035] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:30.405035 3504 consensus_queue.cc:1035] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:30.412281 3499 tablet_replica.cc:439] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd: TxnStatusTablet state changed. Reason: New leader 62cc8fd91a0544cebe5825c9d9651c6e. Latest consensus state: current_term: 1 leader_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } }
I20260810 06:37:30.412592 3499 tablet_replica.cc:442] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:37:30.417013 3535 mvcc.cc:205] Tried to move back new op lower bound from 7316864411242942464 to 7316864411105288192. Current Snapshot: MvccSnapshot[applied={T|T < 7316864411242942464}]
I20260810 06:37:30.417133 3498 tablet_replica.cc:439] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30: TxnStatusTablet state changed. Reason: New leader 62cc8fd91a0544cebe5825c9d9651c6e. Latest consensus state: current_term: 1 leader_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } }
I20260810 06:37:30.417405 3498 tablet_replica.cc:442] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:37:30.418565 3536 mvcc.cc:205] Tried to move back new op lower bound from 7316864411242942464 to 7316864411105288192. Current Snapshot: MvccSnapshot[applied={T|T < 7316864411242942464}]
I20260810 06:37:30.421566 3502 tablet_replica.cc:439] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } }
I20260810 06:37:30.421854 3502 tablet_replica.cc:442] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:37:30.424392 3504 tablet_replica.cc:439] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } }
I20260810 06:37:30.424657 3504 tablet_replica.cc:442] T 4db18e6a17ce42c6bf93532bd09c7111 P 62cc8fd91a0544cebe5825c9d9651c6e: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:37:30.427073 3498 tablet_replica.cc:439] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } }
I20260810 06:37:30.427291 3498 tablet_replica.cc:442] T 4db18e6a17ce42c6bf93532bd09c7111 P e4125072468246afa375da1a22c5be30: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:37:30.428508 3499 tablet_replica.cc:439] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } } }
I20260810 06:37:30.428763 3499 tablet_replica.cc:442] T 4db18e6a17ce42c6bf93532bd09c7111 P c083e39d169f46c7b8e79b96aa62e3dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:37:30.439443 3485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:30.432453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43224 (local address 127.31.59.195:44383)
0810 06:37:30.432605 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:30.432610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:30.434833 (+ 2223us) server_negotiation.cc:407] Connection header received
0810 06:37:30.435009 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:30.435013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:30.435085 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:30.435158 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:37:30.435633 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.436427 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.437766 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.438170 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.438283 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:30.438697 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:37:30.438807 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:37:30.438885 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:30.439051 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 06:37:30.439143 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:37:30.457098 3485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:30.451643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:45863 (local address 127.31.59.195:44383)
0810 06:37:30.451785 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:30.451789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:30.452434 (+ 645us) server_negotiation.cc:407] Connection header received
0810 06:37:30.452625 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:30.452628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:30.452686 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:30.452754 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:37:30.453226 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.453961 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.455419 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.455854 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.455961 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:30.456395 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:37:30.456528 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:37:30.456599 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:30.456720 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 06:37:30.456791 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:37:30.457293 3555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:30.451746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:44383 (local address 127.31.59.195:45863)
0810 06:37:30.452343 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:30.452359 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:30.452495 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:30.452779 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:30.452783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:30.452797 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:37:30.453094 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:30.453102 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.454106 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.454111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:30.455281 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:30.455294 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:30.456032 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:30.456039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:30.456159 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:30.456214 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:37:30.456618 (+ 404us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:37:30.456629 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 06:37:30.456695 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 06:37:30.456826 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":1,"thread_start_us":306,"threads_started":1}
I20260810 06:37:30.467144 2887 raft_consensus.cc:1261] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30 [term 1 FOLLOWER]: Refusing update from remote peer 62cc8fd91a0544cebe5825c9d9651c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:30.467186 3115 raft_consensus.cc:1261] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Refusing update from remote peer 62cc8fd91a0544cebe5825c9d9651c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:30.467805 3504 consensus_queue.cc:1035] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:30.468189 3502 consensus_queue.cc:1035] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:30.483229 3539 mvcc.cc:205] Tried to move back new op lower bound from 7316864411508035584 to 7316864410275815424. Current Snapshot: MvccSnapshot[applied={T|T < 7316864411508035584}]
I20260810 06:37:30.531289 2573 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:37:30.531530 2573 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:37:30.534937 2573 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9: Sending DeleteTablet for 3 replicas of tablet 75820b092dca4b0685bfa064fe46954f
I20260810 06:37:30.536064 3336 tablet_service.cc:1514] Processing DeleteTablet for tablet 75820b092dca4b0685bfa064fe46954f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:30 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:43202
I20260810 06:37:30.536216 3076 tablet_service.cc:1514] Processing DeleteTablet for tablet 75820b092dca4b0685bfa064fe46954f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:30 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:48122
I20260810 06:37:30.537160 2843 tablet_service.cc:1514] Processing DeleteTablet for tablet 75820b092dca4b0685bfa064fe46954f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:30 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:52006
I20260810 06:37:30.537313 2573 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:37:30.538012 2573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:37:30.539582 3569 tablet_replica.cc:330] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd: stopping tablet replica
I20260810 06:37:30.540126 3569 raft_consensus.cc:2229] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:37:30.541159 3567 tablet_replica.cc:330] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e: stopping tablet replica
I20260810 06:37:30.541600 3567 raft_consensus.cc:2229] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:37:30.542322 3567 raft_consensus.cc:2258] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:30.543092 3569 raft_consensus.cc:2258] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:30.545955 3336 tablet_service.cc:1467] Processing CreateTablet for tablet 914b66579015443b9916cd51dfc66ce3 (DEFAULT_TABLE table=test-table [id=66c1d1e68afc4e1ca6ddbc4ec3277ac1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:30.546128 3076 tablet_service.cc:1467] Processing CreateTablet for tablet 914b66579015443b9916cd51dfc66ce3 (DEFAULT_TABLE table=test-table [id=66c1d1e68afc4e1ca6ddbc4ec3277ac1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:30.546363 2842 tablet_service.cc:1467] Processing CreateTablet for tablet 914b66579015443b9916cd51dfc66ce3 (DEFAULT_TABLE table=test-table [id=66c1d1e68afc4e1ca6ddbc4ec3277ac1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:30.546598 3076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 914b66579015443b9916cd51dfc66ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:30.546710 3336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 914b66579015443b9916cd51dfc66ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:30.546868 2842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 914b66579015443b9916cd51dfc66ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:30.553061 3491 tablet_bootstrap.cc:492] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30: Bootstrap starting.
I20260810 06:37:30.553946 3494 tablet_bootstrap.cc:492] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd: Bootstrap starting.
I20260810 06:37:30.554754 3491 tablet_bootstrap.cc:654] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:30.555500 3494 tablet_bootstrap.cc:654] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:30.555464 3567 ts_tablet_manager.cc:1905] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:30.562116 3569 ts_tablet_manager.cc:1905] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:30.563174 3492 tablet_bootstrap.cc:492] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e: Bootstrap starting.
I20260810 06:37:30.564827 3492 tablet_bootstrap.cc:654] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:30.566442 3494 tablet_bootstrap.cc:492] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd: No bootstrap required, opened a new log
I20260810 06:37:30.566630 3494 ts_tablet_manager.cc:1397] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.006s
I20260810 06:37:30.567375 3494 raft_consensus.cc:348] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:30.567628 3494 raft_consensus.cc:374] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:30.567726 3494 raft_consensus.cc:729] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c083e39d169f46c7b8e79b96aa62e3dd, State: Initialized, Role: FOLLOWER
I20260810 06:37:30.568040 3494 consensus_queue.cc:260] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:30.569581 3494 ts_tablet_manager.cc:1428] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260810 06:37:30.571708 3492 tablet_bootstrap.cc:492] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e: No bootstrap required, opened a new log
I20260810 06:37:30.571870 3492 ts_tablet_manager.cc:1397] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.004s
I20260810 06:37:30.572444 3491 tablet_bootstrap.cc:492] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30: No bootstrap required, opened a new log
I20260810 06:37:30.572642 3491 ts_tablet_manager.cc:1397] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30: Time spent bootstrapping tablet: real 0.020s user 0.012s sys 0.000s
I20260810 06:37:30.572600 3492 raft_consensus.cc:348] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:30.572746 3570 tablet_replica.cc:330] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30: stopping tablet replica
I20260810 06:37:30.572829 3492 raft_consensus.cc:374] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:30.572907 3492 raft_consensus.cc:729] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62cc8fd91a0544cebe5825c9d9651c6e, State: Initialized, Role: FOLLOWER
I20260810 06:37:30.573139 3492 consensus_queue.cc:260] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:30.573383 3491 raft_consensus.cc:348] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:30.573635 3491 raft_consensus.cc:374] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:30.573684 3492 ts_tablet_manager.cc:1428] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 06:37:30.573705 3491 raft_consensus.cc:729] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4125072468246afa375da1a22c5be30, State: Initialized, Role: FOLLOWER
I20260810 06:37:30.573926 3491 consensus_queue.cc:260] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:30.574898 3570 raft_consensus.cc:2229] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:37:30.575307 3570 raft_consensus.cc:2258] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:30.575410 3491 ts_tablet_manager.cc:1428] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:37:30.577212 3569 ts_tablet_manager.cc:1918] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:37:30.577380 3569 log.cc:1196] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/75820b092dca4b0685bfa064fe46954f
I20260810 06:37:30.577958 3569 ts_tablet_manager.cc:1939] T 75820b092dca4b0685bfa064fe46954f P c083e39d169f46c7b8e79b96aa62e3dd: Deleting consensus metadata
I20260810 06:37:30.579785 3567 ts_tablet_manager.cc:1918] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:37:30.580034 3567 log.cc:1196] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/75820b092dca4b0685bfa064fe46954f
I20260810 06:37:30.580380 2548 catalog_manager.cc:4841] TS c083e39d169f46c7b8e79b96aa62e3dd (127.31.59.194:39235): tablet 75820b092dca4b0685bfa064fe46954f (table test-table [id=339706564fef4ca5bdf755fde001d849]) successfully deleted
I20260810 06:37:30.580682 3567 ts_tablet_manager.cc:1939] T 75820b092dca4b0685bfa064fe46954f P 62cc8fd91a0544cebe5825c9d9651c6e: Deleting consensus metadata
I20260810 06:37:30.582120 3570 ts_tablet_manager.cc:1905] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:30.582254 2549 catalog_manager.cc:4841] TS 62cc8fd91a0544cebe5825c9d9651c6e (127.31.59.195:44383): tablet 75820b092dca4b0685bfa064fe46954f (table test-table [id=339706564fef4ca5bdf755fde001d849]) successfully deleted
I20260810 06:37:30.587275 3570 ts_tablet_manager.cc:1918] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:37:30.587455 3570 log.cc:1196] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/75820b092dca4b0685bfa064fe46954f
I20260810 06:37:30.587993 3570 ts_tablet_manager.cc:1939] T 75820b092dca4b0685bfa064fe46954f P e4125072468246afa375da1a22c5be30: Deleting consensus metadata
I20260810 06:37:30.589421 2550 catalog_manager.cc:4841] TS e4125072468246afa375da1a22c5be30 (127.31.59.193:35223): tablet 75820b092dca4b0685bfa064fe46954f (table test-table [id=339706564fef4ca5bdf755fde001d849]) successfully deleted
I20260810 06:37:30.646709 3561 raft_consensus.cc:482] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:37:30.646927 3561 raft_consensus.cc:504] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:30.647780 3561 leader_election.cc:288] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4125072468246afa375da1a22c5be30 (127.31.59.193:35223), c083e39d169f46c7b8e79b96aa62e3dd (127.31.59.194:39235)
I20260810 06:37:30.648397 3116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "914b66579015443b9916cd51dfc66ce3" candidate_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" is_pre_election: true
I20260810 06:37:30.648358 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "914b66579015443b9916cd51dfc66ce3" candidate_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4125072468246afa375da1a22c5be30" is_pre_election: true
I20260810 06:37:30.648646 2884 raft_consensus.cc:2454] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62cc8fd91a0544cebe5825c9d9651c6e in term 0.
I20260810 06:37:30.648666 3116 raft_consensus.cc:2454] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62cc8fd91a0544cebe5825c9d9651c6e in term 0.
I20260810 06:37:30.649101 3241 leader_election.cc:302] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62cc8fd91a0544cebe5825c9d9651c6e, c083e39d169f46c7b8e79b96aa62e3dd; no voters:
I20260810 06:37:30.649400 3561 raft_consensus.cc:2781] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:37:30.649544 3561 raft_consensus.cc:482] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:37:30.649638 3561 raft_consensus.cc:3037] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:30.653035 3561 raft_consensus.cc:504] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:30.653705 3561 leader_election.cc:288] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [CANDIDATE]: Term 1 election: Requested vote from peers e4125072468246afa375da1a22c5be30 (127.31.59.193:35223), c083e39d169f46c7b8e79b96aa62e3dd (127.31.59.194:39235)
I20260810 06:37:30.654202 3116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "914b66579015443b9916cd51dfc66ce3" candidate_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c083e39d169f46c7b8e79b96aa62e3dd"
I20260810 06:37:30.654310 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "914b66579015443b9916cd51dfc66ce3" candidate_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4125072468246afa375da1a22c5be30"
I20260810 06:37:30.654466 3116 raft_consensus.cc:3037] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:30.654518 2884 raft_consensus.cc:3037] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:30.657141 2884 raft_consensus.cc:2454] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62cc8fd91a0544cebe5825c9d9651c6e in term 1.
I20260810 06:37:30.657321 3116 raft_consensus.cc:2454] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62cc8fd91a0544cebe5825c9d9651c6e in term 1.
I20260810 06:37:30.657562 3244 leader_election.cc:302] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62cc8fd91a0544cebe5825c9d9651c6e, e4125072468246afa375da1a22c5be30; no voters:
I20260810 06:37:30.657847 3561 raft_consensus.cc:2781] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:30.658000 3561 raft_consensus.cc:686] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 LEADER]: Becoming Leader. State: Replica: 62cc8fd91a0544cebe5825c9d9651c6e, State: Running, Role: LEADER
I20260810 06:37:30.658257 3561 consensus_queue.cc:237] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:30.660871 2575 catalog_manager.cc:5495] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 62cc8fd91a0544cebe5825c9d9651c6e (127.31.59.195). New cstate: current_term: 1 leader_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } health_report { overall_health: HEALTHY } } }
I20260810 06:37:30.700843 2884 raft_consensus.cc:1261] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30 [term 1 FOLLOWER]: Refusing update from remote peer 62cc8fd91a0544cebe5825c9d9651c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:30.701390 3561 consensus_queue.cc:1035] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:30.701558 3116 raft_consensus.cc:1261] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Refusing update from remote peer 62cc8fd91a0544cebe5825c9d9651c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:30.702056 3561 consensus_queue.cc:1035] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 06:37:30.727737 3535 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 06:37:32.315217 2573 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:37:32.315439 2573 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:37:32.320287 2573 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9: Sending DeleteTablet for 3 replicas of tablet 914b66579015443b9916cd51dfc66ce3
I20260810 06:37:32.321152 3076 tablet_service.cc:1514] Processing DeleteTablet for tablet 914b66579015443b9916cd51dfc66ce3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:32 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:48122
I20260810 06:37:32.321234 2843 tablet_service.cc:1514] Processing DeleteTablet for tablet 914b66579015443b9916cd51dfc66ce3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:32 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:52006
I20260810 06:37:32.321224 3336 tablet_service.cc:1514] Processing DeleteTablet for tablet 914b66579015443b9916cd51dfc66ce3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:32 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:43202
I20260810 06:37:32.335392 3630 tablet_replica.cc:330] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e: stopping tablet replica
I20260810 06:37:32.335613 3630 raft_consensus.cc:2229] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:37:32.335981 3630 raft_consensus.cc:2258] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:32.336524 3625 tablet_replica.cc:330] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd: stopping tablet replica
I20260810 06:37:32.336790 3625 raft_consensus.cc:2229] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:37:32.337039 3625 raft_consensus.cc:2258] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:32.337877 3630 ts_tablet_manager.cc:1905] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:32.339578 3633 tablet_replica.cc:330] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30: stopping tablet replica
I20260810 06:37:32.339762 3633 raft_consensus.cc:2229] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:37:32.339983 3633 raft_consensus.cc:2258] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:32.340010 3625 ts_tablet_manager.cc:1905] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:32.342449 3633 ts_tablet_manager.cc:1905] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:32.345345 3625 ts_tablet_manager.cc:1918] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:37:32.345474 3625 log.cc:1196] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/914b66579015443b9916cd51dfc66ce3
I20260810 06:37:32.345872 3625 ts_tablet_manager.cc:1939] T 914b66579015443b9916cd51dfc66ce3 P c083e39d169f46c7b8e79b96aa62e3dd: Deleting consensus metadata
I20260810 06:37:32.347020 2548 catalog_manager.cc:4841] TS c083e39d169f46c7b8e79b96aa62e3dd (127.31.59.194:39235): tablet 914b66579015443b9916cd51dfc66ce3 (table test-table [id=66c1d1e68afc4e1ca6ddbc4ec3277ac1]) successfully deleted
I20260810 06:37:32.347250 3630 ts_tablet_manager.cc:1918] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:37:32.347429 3630 log.cc:1196] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/914b66579015443b9916cd51dfc66ce3
I20260810 06:37:32.347888 3630 ts_tablet_manager.cc:1939] T 914b66579015443b9916cd51dfc66ce3 P 62cc8fd91a0544cebe5825c9d9651c6e: Deleting consensus metadata
I20260810 06:37:32.348999 2549 catalog_manager.cc:4841] TS 62cc8fd91a0544cebe5825c9d9651c6e (127.31.59.195:44383): tablet 914b66579015443b9916cd51dfc66ce3 (table test-table [id=66c1d1e68afc4e1ca6ddbc4ec3277ac1]) successfully deleted
I20260810 06:37:32.350132 3633 ts_tablet_manager.cc:1918] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:37:32.350272 3633 log.cc:1196] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/914b66579015443b9916cd51dfc66ce3
I20260810 06:37:32.350731 3633 ts_tablet_manager.cc:1939] T 914b66579015443b9916cd51dfc66ce3 P e4125072468246afa375da1a22c5be30: Deleting consensus metadata
I20260810 06:37:32.351648 2550 catalog_manager.cc:4841] TS e4125072468246afa375da1a22c5be30 (127.31.59.193:35223): tablet 914b66579015443b9916cd51dfc66ce3 (table test-table [id=66c1d1e68afc4e1ca6ddbc4ec3277ac1]) successfully deleted
Aug 10 06:37:32 dist-test-slave-4tdj krb5kdc[2495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343852, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:37:32 dist-test-slave-4tdj krb5kdc[2495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343852, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:32.383406 3647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:32.367054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42884 (local address 127.31.59.254:38471)
0810 06:37:32.367589 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:32.367595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:32.367934 (+ 339us) server_negotiation.cc:407] Connection header received
0810 06:37:32.368153 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:32.368158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:32.368238 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:32.368340 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:32.369840 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.371072 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.372672 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.373300 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.373435 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:32.377869 (+ 4434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:32.377876 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:32.377881 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:32.377921 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:32.381061 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:32.381694 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:32.381698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:32.381700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:32.381809 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:32.382124 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:32.382127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:32.382129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:32.382421 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:32.382610 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:32.382893 (+ 283us) server_negotiation.cc:299] Negotiation successful
0810 06:37:32.383071 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":235,"threads_started":1}
I20260810 06:37:32.387224 2573 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:37:32.387804 2573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:37:32.394353 2843 tablet_service.cc:1467] Processing CreateTablet for tablet 952224d087224bada4996fe93f70bd9b (DEFAULT_TABLE table=test-table [id=3b72411a7984457094d452e4a614b084]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:32.394389 3076 tablet_service.cc:1467] Processing CreateTablet for tablet 952224d087224bada4996fe93f70bd9b (DEFAULT_TABLE table=test-table [id=3b72411a7984457094d452e4a614b084]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:32.394506 3336 tablet_service.cc:1467] Processing CreateTablet for tablet 952224d087224bada4996fe93f70bd9b (DEFAULT_TABLE table=test-table [id=3b72411a7984457094d452e4a614b084]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:32.394872 2843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 952224d087224bada4996fe93f70bd9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:32.394994 3076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 952224d087224bada4996fe93f70bd9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:32.395088 3336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 952224d087224bada4996fe93f70bd9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:32.401443 3648 tablet_bootstrap.cc:492] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30: Bootstrap starting.
I20260810 06:37:32.402045 3651 tablet_bootstrap.cc:492] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e: Bootstrap starting.
I20260810 06:37:32.403769 3648 tablet_bootstrap.cc:654] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:32.404083 3651 tablet_bootstrap.cc:654] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:32.405467 3649 tablet_bootstrap.cc:492] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd: Bootstrap starting.
I20260810 06:37:32.406800 3648 tablet_bootstrap.cc:492] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30: No bootstrap required, opened a new log
I20260810 06:37:32.407023 3648 ts_tablet_manager.cc:1397] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 06:37:32.407199 3651 tablet_bootstrap.cc:492] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e: No bootstrap required, opened a new log
I20260810 06:37:32.407231 3649 tablet_bootstrap.cc:654] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:32.407410 3651 ts_tablet_manager.cc:1397] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260810 06:37:32.408749 3651 raft_consensus.cc:348] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.408985 3648 raft_consensus.cc:348] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.409174 3651 raft_consensus.cc:374] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:32.409265 3648 raft_consensus.cc:374] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:32.409413 3651 raft_consensus.cc:729] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62cc8fd91a0544cebe5825c9d9651c6e, State: Initialized, Role: FOLLOWER
I20260810 06:37:32.409540 3648 raft_consensus.cc:729] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4125072468246afa375da1a22c5be30, State: Initialized, Role: FOLLOWER
I20260810 06:37:32.409679 3651 consensus_queue.cc:260] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.409936 3648 consensus_queue.cc:260] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.410609 3649 tablet_bootstrap.cc:492] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd: No bootstrap required, opened a new log
I20260810 06:37:32.410758 3649 ts_tablet_manager.cc:1397] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 06:37:32.410912 3648 ts_tablet_manager.cc:1428] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260810 06:37:32.411495 3649 raft_consensus.cc:348] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.411717 3649 raft_consensus.cc:374] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:32.411815 3649 raft_consensus.cc:729] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c083e39d169f46c7b8e79b96aa62e3dd, State: Initialized, Role: FOLLOWER
I20260810 06:37:32.412062 3649 consensus_queue.cc:260] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.412305 3651 ts_tablet_manager.cc:1428] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260810 06:37:32.414867 3649 ts_tablet_manager.cc:1428] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd: Time spent starting tablet: real 0.004s user 0.002s sys 0.003s
I20260810 06:37:32.610127 3658 raft_consensus.cc:482] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:37:32.610566 3658 raft_consensus.cc:504] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.612289 3658 leader_election.cc:288] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c083e39d169f46c7b8e79b96aa62e3dd (127.31.59.194:39235), 62cc8fd91a0544cebe5825c9d9651c6e (127.31.59.195:44383)
I20260810 06:37:32.619935 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:32.612713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:39615 (local address 127.31.59.194:39235)
0810 06:37:32.613412 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:32.613418 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:32.614169 (+ 751us) server_negotiation.cc:407] Connection header received
0810 06:37:32.614441 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:32.614445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:32.614570 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:32.614665 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:32.615446 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.616216 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.618065 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.618912 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.619161 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:32.619236 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:32.619522 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 06:37:32.619658 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":390,"threads_started":1}
I20260810 06:37:32.619935 3665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:32.612708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:50797 (local address 127.31.59.195:44383)
0810 06:37:32.614244 (+ 1536us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:32.614250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:32.614274 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:37:32.614383 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:32.614388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:32.614466 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:32.614551 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:32.615381 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.616210 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.617987 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.618840 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.619024 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:32.619111 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:32.619388 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 06:37:32.619519 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1404,"thread_start_us":1232,"threads_started":1}
I20260810 06:37:32.620704 3669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:32.612825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:39235 (local address 127.31.59.193:39615)
0810 06:37:32.614061 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:32.614079 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:32.614251 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:32.614707 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:32.614711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:32.614751 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:32.615273 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:32.615283 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.616344 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.616349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:32.617930 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:32.617945 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.619212 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.619218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:32.619375 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:32.619397 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 06:37:32.619474 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 06:37:32.619592 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1062,"mutex_wait_us":39,"spinlock_wait_cycles":47488,"thread_start_us":813,"threads_started":1}
I20260810 06:37:32.620823 3671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:32.612801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:44383 (local address 127.31.59.193:50797)
0810 06:37:32.613854 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:32.613876 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:32.614027 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:32.614591 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:32.614596 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:32.614621 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:32.615209 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:32.615223 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.616338 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.616343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:32.617819 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:32.617839 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.619027 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.619034 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:32.619221 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:32.619272 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 06:37:32.619351 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 06:37:32.619473 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":870,"mutex_wait_us":43,"thread_start_us":538,"threads_started":1}
I20260810 06:37:32.621423 3115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "952224d087224bada4996fe93f70bd9b" candidate_uuid: "e4125072468246afa375da1a22c5be30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" is_pre_election: true
I20260810 06:37:32.621745 3115 raft_consensus.cc:2454] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4125072468246afa375da1a22c5be30 in term 0.
I20260810 06:37:32.621657 3377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "952224d087224bada4996fe93f70bd9b" candidate_uuid: "e4125072468246afa375da1a22c5be30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" is_pre_election: true
I20260810 06:37:32.622010 3377 raft_consensus.cc:2454] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4125072468246afa375da1a22c5be30 in term 0.
I20260810 06:37:32.622385 2753 leader_election.cc:302] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c083e39d169f46c7b8e79b96aa62e3dd, e4125072468246afa375da1a22c5be30; no voters:
I20260810 06:37:32.622789 3658 raft_consensus.cc:2781] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:37:32.622915 3658 raft_consensus.cc:482] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:37:32.623009 3658 raft_consensus.cc:3037] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:32.625272 3658 raft_consensus.cc:504] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.625852 3658 leader_election.cc:288] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [CANDIDATE]: Term 1 election: Requested vote from peers c083e39d169f46c7b8e79b96aa62e3dd (127.31.59.194:39235), 62cc8fd91a0544cebe5825c9d9651c6e (127.31.59.195:44383)
I20260810 06:37:32.626226 3115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "952224d087224bada4996fe93f70bd9b" candidate_uuid: "e4125072468246afa375da1a22c5be30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c083e39d169f46c7b8e79b96aa62e3dd"
I20260810 06:37:32.626335 3377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "952224d087224bada4996fe93f70bd9b" candidate_uuid: "e4125072468246afa375da1a22c5be30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62cc8fd91a0544cebe5825c9d9651c6e"
I20260810 06:37:32.626494 3115 raft_consensus.cc:3037] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:32.626503 3377 raft_consensus.cc:3037] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:32.628594 3115 raft_consensus.cc:2454] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4125072468246afa375da1a22c5be30 in term 1.
I20260810 06:37:32.628958 2753 leader_election.cc:302] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c083e39d169f46c7b8e79b96aa62e3dd, e4125072468246afa375da1a22c5be30; no voters:
I20260810 06:37:32.629247 3658 raft_consensus.cc:2781] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:32.629319 3377 raft_consensus.cc:2454] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4125072468246afa375da1a22c5be30 in term 1.
I20260810 06:37:32.629736 3658 raft_consensus.cc:686] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [term 1 LEADER]: Becoming Leader. State: Replica: e4125072468246afa375da1a22c5be30, State: Running, Role: LEADER
I20260810 06:37:32.630134 3658 consensus_queue.cc:237] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.633366 2573 catalog_manager.cc:5495] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 reported cstate change: term changed from 0 to 1, leader changed from <none> to e4125072468246afa375da1a22c5be30 (127.31.59.193). New cstate: current_term: 1 leader_uuid: "e4125072468246afa375da1a22c5be30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } health_report { overall_health: UNKNOWN } } }
I20260810 06:37:32.708029 3676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:32.702026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52014 (local address 127.31.59.193:35223)
0810 06:37:32.702579 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:32.702583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:32.702600 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:37:32.702680 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:32.702683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:32.702755 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:32.702901 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:37:32.703337 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.704122 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.705797 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.706208 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.706342 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:32.706791 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:37:32.707366 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:37:32.707503 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:32.707623 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 06:37:32.707712 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":307,"threads_started":1}
I20260810 06:37:32.725807 3665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:32.719355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:36895 (local address 127.31.59.195:44383)
0810 06:37:32.719554 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:32.719560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:32.720747 (+ 1187us) server_negotiation.cc:407] Connection header received
0810 06:37:32.720927 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:32.720932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:32.721010 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:32.721081 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:37:32.721598 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.722486 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.724120 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.724530 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.724632 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:32.725095 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:37:32.725188 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:37:32.725271 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:32.725396 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 06:37:32.725469 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:37:32.725874 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:32.719463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:44383 (local address 127.31.59.193:36895)
0810 06:37:32.720654 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:32.720669 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:32.720799 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:32.721107 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:32.721111 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:32.721123 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:37:32.721457 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:32.721466 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.722614 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.722620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:32.723992 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:32.724005 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.724744 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.724751 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:32.724873 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:32.724925 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:37:32.725287 (+ 362us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:37:32.725299 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 06:37:32.725366 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 06:37:32.725500 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1001,"thread_start_us":827,"threads_started":1}
I20260810 06:37:32.735109 3377 raft_consensus.cc:1261] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 FOLLOWER]: Refusing update from remote peer e4125072468246afa375da1a22c5be30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:32.735320 3115 raft_consensus.cc:1261] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Refusing update from remote peer e4125072468246afa375da1a22c5be30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:32.735769 3672 consensus_queue.cc:1035] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:32.736249 3658 consensus_queue.cc:1035] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:32.772457 3676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:32.762429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:47455 (local address 127.31.59.193:35223)
0810 06:37:32.762579 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:32.762583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:32.766950 (+ 4367us) server_negotiation.cc:407] Connection header received
0810 06:37:32.767130 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:32.767136 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:32.767198 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:32.767269 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:37:32.767817 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.769107 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.770604 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.771111 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.771194 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:32.771749 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:37:32.771859 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:37:32.771911 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:32.772112 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 06:37:32.772183 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:37:32.773499 3686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:32.762203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:35223 (local address 127.31.59.195:47455)
0810 06:37:32.766851 (+ 4648us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:32.766866 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:32.766992 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:32.767296 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:32.767300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:32.767311 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:37:32.767658 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:32.767667 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.769226 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.769231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:32.770448 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:32.770463 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.771375 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.771380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:32.771541 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:32.771584 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:37:32.771982 (+ 398us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:37:32.771991 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 06:37:32.772715 (+ 724us) client_negotiation.cc:240] Negotiation successful
0810 06:37:32.772836 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4506,"thread_start_us":4299,"threads_started":1}
I20260810 06:37:32.810102 2573 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:37:32.810441 2573 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:37:32.814265 2573 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9: Sending DeleteTablet for 3 replicas of tablet 952224d087224bada4996fe93f70bd9b
I20260810 06:37:32.815164 2843 tablet_service.cc:1514] Processing DeleteTablet for tablet 952224d087224bada4996fe93f70bd9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:32 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:52006
I20260810 06:37:32.815549 3633 tablet_replica.cc:330] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30: stopping tablet replica
I20260810 06:37:32.815975 3633 raft_consensus.cc:2229] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:37:32.816195 3076 tablet_service.cc:1514] Processing DeleteTablet for tablet 952224d087224bada4996fe93f70bd9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:32 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:48122
I20260810 06:37:32.816490 3625 tablet_replica.cc:330] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd: stopping tablet replica
I20260810 06:37:32.816565 3633 raft_consensus.cc:2258] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:32.816684 3625 raft_consensus.cc:2229] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:37:32.816874 3625 raft_consensus.cc:2258] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:32.818454 3625 ts_tablet_manager.cc:1905] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:32.818585 2573 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:37:32.819238 2573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:37:32.823061 3633 ts_tablet_manager.cc:1905] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:32.827369 3076 tablet_service.cc:1467] Processing CreateTablet for tablet 71e2e348391c4b21b4cc3494bbe4a13b (DEFAULT_TABLE table=test-table [id=e5b7407981ff4be39da2182b2ac5d94c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:32.827452 3335 tablet_service.cc:1467] Processing CreateTablet for tablet 71e2e348391c4b21b4cc3494bbe4a13b (DEFAULT_TABLE table=test-table [id=e5b7407981ff4be39da2182b2ac5d94c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:32.827610 2843 tablet_service.cc:1467] Processing CreateTablet for tablet 71e2e348391c4b21b4cc3494bbe4a13b (DEFAULT_TABLE table=test-table [id=e5b7407981ff4be39da2182b2ac5d94c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:32.827869 3335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71e2e348391c4b21b4cc3494bbe4a13b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:32.827899 3076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71e2e348391c4b21b4cc3494bbe4a13b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:32.828012 2843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71e2e348391c4b21b4cc3494bbe4a13b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:32.834364 3651 tablet_bootstrap.cc:492] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e: Bootstrap starting.
I20260810 06:37:32.834694 3336 tablet_service.cc:1514] Processing DeleteTablet for tablet 952224d087224bada4996fe93f70bd9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:32 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:43202
I20260810 06:37:32.835029 3630 tablet_replica.cc:330] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e: stopping tablet replica
I20260810 06:37:32.835245 3630 raft_consensus.cc:2229] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:37:32.835445 3630 raft_consensus.cc:2258] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:32.836311 3651 tablet_bootstrap.cc:654] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:32.838397 3630 ts_tablet_manager.cc:1905] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:32.843677 3625 ts_tablet_manager.cc:1918] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:37:32.843888 3625 log.cc:1196] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/952224d087224bada4996fe93f70bd9b
I20260810 06:37:32.844381 3625 ts_tablet_manager.cc:1939] T 952224d087224bada4996fe93f70bd9b P c083e39d169f46c7b8e79b96aa62e3dd: Deleting consensus metadata
I20260810 06:37:32.845507 2548 catalog_manager.cc:4841] TS c083e39d169f46c7b8e79b96aa62e3dd (127.31.59.194:39235): tablet 952224d087224bada4996fe93f70bd9b (table test-table [id=3b72411a7984457094d452e4a614b084]) successfully deleted
I20260810 06:37:32.846725 3649 tablet_bootstrap.cc:492] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd: Bootstrap starting.
I20260810 06:37:32.848771 3649 tablet_bootstrap.cc:654] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:32.849571 3633 ts_tablet_manager.cc:1918] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:37:32.849750 3633 log.cc:1196] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/952224d087224bada4996fe93f70bd9b
I20260810 06:37:32.850180 3633 ts_tablet_manager.cc:1939] T 952224d087224bada4996fe93f70bd9b P e4125072468246afa375da1a22c5be30: Deleting consensus metadata
I20260810 06:37:32.851562 3630 ts_tablet_manager.cc:1918] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:37:32.851907 3648 tablet_bootstrap.cc:492] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30: Bootstrap starting.
I20260810 06:37:32.853734 3648 tablet_bootstrap.cc:654] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:32.856158 2550 catalog_manager.cc:4841] TS e4125072468246afa375da1a22c5be30 (127.31.59.193:35223): tablet 952224d087224bada4996fe93f70bd9b (table test-table [id=3b72411a7984457094d452e4a614b084]) successfully deleted
I20260810 06:37:32.858059 3651 tablet_bootstrap.cc:492] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e: No bootstrap required, opened a new log
I20260810 06:37:32.858923 3651 ts_tablet_manager.cc:1397] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.002s
I20260810 06:37:32.860015 3651 raft_consensus.cc:348] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.860436 3651 raft_consensus.cc:374] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:32.860584 3651 raft_consensus.cc:729] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62cc8fd91a0544cebe5825c9d9651c6e, State: Initialized, Role: FOLLOWER
I20260810 06:37:32.861052 3651 consensus_queue.cc:260] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.861898 3651 ts_tablet_manager.cc:1428] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260810 06:37:32.862540 3630 log.cc:1196] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/952224d087224bada4996fe93f70bd9b
I20260810 06:37:32.863399 3630 ts_tablet_manager.cc:1939] T 952224d087224bada4996fe93f70bd9b P 62cc8fd91a0544cebe5825c9d9651c6e: Deleting consensus metadata
I20260810 06:37:32.865731 2549 catalog_manager.cc:4841] TS 62cc8fd91a0544cebe5825c9d9651c6e (127.31.59.195:44383): tablet 952224d087224bada4996fe93f70bd9b (table test-table [id=3b72411a7984457094d452e4a614b084]) successfully deleted
I20260810 06:37:32.870564 3649 tablet_bootstrap.cc:492] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd: No bootstrap required, opened a new log
I20260810 06:37:32.870743 3649 ts_tablet_manager.cc:1397] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.007s
I20260810 06:37:32.870873 3648 tablet_bootstrap.cc:492] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30: No bootstrap required, opened a new log
I20260810 06:37:32.871093 3648 ts_tablet_manager.cc:1397] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.007s
I20260810 06:37:32.871510 3649 raft_consensus.cc:348] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.871767 3649 raft_consensus.cc:374] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:32.871856 3649 raft_consensus.cc:729] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c083e39d169f46c7b8e79b96aa62e3dd, State: Initialized, Role: FOLLOWER
I20260810 06:37:32.872043 3648 raft_consensus.cc:348] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.872143 3649 consensus_queue.cc:260] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.872426 3648 raft_consensus.cc:374] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:32.872670 3648 raft_consensus.cc:729] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4125072468246afa375da1a22c5be30, State: Initialized, Role: FOLLOWER
I20260810 06:37:32.873064 3648 consensus_queue.cc:260] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.874182 3648 ts_tablet_manager.cc:1428] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:37:32.875046 3649 ts_tablet_manager.cc:1428] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260810 06:37:32.915944 3659 raft_consensus.cc:482] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:37:32.916224 3659 raft_consensus.cc:504] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.918216 3659 leader_election.cc:288] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4125072468246afa375da1a22c5be30 (127.31.59.193:35223), 62cc8fd91a0544cebe5825c9d9651c6e (127.31.59.195:44383)
I20260810 06:37:32.920497 3594 raft_consensus.cc:482] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:37:32.920753 3594 raft_consensus.cc:504] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.921509 3594 leader_election.cc:288] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4125072468246afa375da1a22c5be30 (127.31.59.193:35223), c083e39d169f46c7b8e79b96aa62e3dd (127.31.59.194:39235)
I20260810 06:37:32.922353 2887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e2e348391c4b21b4cc3494bbe4a13b" candidate_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4125072468246afa375da1a22c5be30" is_pre_election: true
I20260810 06:37:32.922680 2887 raft_consensus.cc:2454] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62cc8fd91a0544cebe5825c9d9651c6e in term 0.
I20260810 06:37:32.923471 3115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e2e348391c4b21b4cc3494bbe4a13b" candidate_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" is_pre_election: true
I20260810 06:37:32.923810 3115 raft_consensus.cc:2454] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62cc8fd91a0544cebe5825c9d9651c6e in term 0.
I20260810 06:37:32.924333 3244 leader_election.cc:302] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62cc8fd91a0544cebe5825c9d9651c6e, e4125072468246afa375da1a22c5be30; no voters:
I20260810 06:37:32.925716 3594 raft_consensus.cc:2781] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:37:32.925858 3594 raft_consensus.cc:482] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:37:32.925959 3594 raft_consensus.cc:3037] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:32.929059 3594 raft_consensus.cc:504] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.930122 2887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e2e348391c4b21b4cc3494bbe4a13b" candidate_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4125072468246afa375da1a22c5be30"
I20260810 06:37:32.930346 2887 raft_consensus.cc:3037] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:32.933987 2887 raft_consensus.cc:2454] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62cc8fd91a0544cebe5825c9d9651c6e in term 1.
I20260810 06:37:32.934444 3244 leader_election.cc:302] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62cc8fd91a0544cebe5825c9d9651c6e, e4125072468246afa375da1a22c5be30; no voters:
I20260810 06:37:32.934556 3676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:32.919713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:41659 (local address 127.31.59.193:35223)
0810 06:37:32.919888 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:32.919895 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:32.925850 (+ 5955us) server_negotiation.cc:407] Connection header received
0810 06:37:32.926148 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:32.926154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:32.926244 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:32.926333 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:32.927502 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.929033 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.931821 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.933288 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.933428 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:32.933502 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:32.934094 (+ 592us) server_negotiation.cc:299] Negotiation successful
0810 06:37:32.934245 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:37:32.934819 3610 raft_consensus.cc:2781] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:32.934976 3610 raft_consensus.cc:686] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 LEADER]: Becoming Leader. State: Replica: 62cc8fd91a0544cebe5825c9d9651c6e, State: Running, Role: LEADER
I20260810 06:37:32.935523 3693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:32.919380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:35223 (local address 127.31.59.194:41659)
0810 06:37:32.925744 (+ 6364us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:32.925765 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:32.925961 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:32.926566 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:32.926571 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:32.926598 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:32.927320 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:32.927335 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.929278 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.929283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:32.931635 (+ 2352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:32.931653 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.933701 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.933708 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:32.933891 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:32.933927 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:37:32.934957 (+ 1030us) client_negotiation.cc:240] Negotiation successful
0810 06:37:32.935190 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6227,"thread_start_us":6066,"threads_started":1}
I20260810 06:37:32.935305 3610 consensus_queue.cc:237] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } }
I20260810 06:37:32.939081 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e2e348391c4b21b4cc3494bbe4a13b" candidate_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4125072468246afa375da1a22c5be30" is_pre_election: true
I20260810 06:37:32.939476 2884 raft_consensus.cc:2379] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c083e39d169f46c7b8e79b96aa62e3dd in current term 1: Already voted for candidate 62cc8fd91a0544cebe5825c9d9651c6e in this term.
I20260810 06:37:32.945130 3665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:32.918856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:41955 (local address 127.31.59.195:44383)
0810 06:37:32.919055 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:32.919062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:32.936065 (+ 17003us) server_negotiation.cc:407] Connection header received
0810 06:37:32.936311 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:32.936317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:32.936393 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:32.936450 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:32.936982 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.938447 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.942515 (+ 4068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.943920 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.944075 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:32.944150 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:32.944700 (+ 550us) server_negotiation.cc:299] Negotiation successful
0810 06:37:32.944838 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:37:32.945334 3115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e2e348391c4b21b4cc3494bbe4a13b" candidate_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c083e39d169f46c7b8e79b96aa62e3dd"
I20260810 06:37:32.945591 3115 raft_consensus.cc:3037] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:32.945992 3693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:32.920115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:44383 (local address 127.31.59.194:41955)
0810 06:37:32.935964 (+ 15849us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:32.935983 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:32.936168 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:32.936637 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:32.936642 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:32.936668 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:32.936832 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:32.936843 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.940338 (+ 3495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.940343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:32.942285 (+ 1942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:32.942320 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.944303 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.944309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:32.944519 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:32.944552 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:37:32.945463 (+ 911us) client_negotiation.cc:240] Negotiation successful
0810 06:37:32.945704 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15686,"threads_started":1}
I20260810 06:37:32.948664 3115 raft_consensus.cc:2454] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62cc8fd91a0544cebe5825c9d9651c6e in term 1.
I20260810 06:37:32.949298 3594 leader_election.cc:288] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [CANDIDATE]: Term 1 election: Requested vote from peers e4125072468246afa375da1a22c5be30 (127.31.59.193:35223), c083e39d169f46c7b8e79b96aa62e3dd (127.31.59.194:39235)
I20260810 06:37:32.955222 2575 catalog_manager.cc:5495] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 62cc8fd91a0544cebe5825c9d9651c6e (127.31.59.195). New cstate: current_term: 1 leader_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44383 } health_report { overall_health: HEALTHY } } }
I20260810 06:37:32.955845 3377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e2e348391c4b21b4cc3494bbe4a13b" candidate_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62cc8fd91a0544cebe5825c9d9651c6e" is_pre_election: true
I20260810 06:37:32.956578 3001 leader_election.cc:302] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c083e39d169f46c7b8e79b96aa62e3dd; no voters: 62cc8fd91a0544cebe5825c9d9651c6e, e4125072468246afa375da1a22c5be30
I20260810 06:37:32.957646 3659 raft_consensus.cc:2726] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 06:37:32.983685 3665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:32.977267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43238 (local address 127.31.59.195:44383)
0810 06:37:32.977505 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:32.977513 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:32.977533 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:37:32.977601 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:32.977607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:32.977693 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:32.977787 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:37:32.978546 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.979776 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.981680 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:32.982367 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:32.982493 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:32.982963 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:37:32.983106 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:37:32.983207 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:32.983335 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 06:37:32.983413 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:37:32.996119 3115 raft_consensus.cc:1261] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Refusing update from remote peer 62cc8fd91a0544cebe5825c9d9651c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:32.996171 2884 raft_consensus.cc:1261] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30 [term 1 FOLLOWER]: Refusing update from remote peer 62cc8fd91a0544cebe5825c9d9651c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:32.996668 3594 consensus_queue.cc:1035] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c083e39d169f46c7b8e79b96aa62e3dd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 39235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:32.997052 3609 consensus_queue.cc:1035] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4125072468246afa375da1a22c5be30" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 06:37:33.031992 3591 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 06:37:34.613757 2576 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:37:34.613991 2576 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:37:34.618582 2576 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55d6012986854c618d7029c11e126db9: Sending DeleteTablet for 3 replicas of tablet 71e2e348391c4b21b4cc3494bbe4a13b
I20260810 06:37:34.619288 2843 tablet_service.cc:1514] Processing DeleteTablet for tablet 71e2e348391c4b21b4cc3494bbe4a13b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:34 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:52006
I20260810 06:37:34.619470 3336 tablet_service.cc:1514] Processing DeleteTablet for tablet 71e2e348391c4b21b4cc3494bbe4a13b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:34 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:43202
I20260810 06:37:34.619550 3076 tablet_service.cc:1514] Processing DeleteTablet for tablet 71e2e348391c4b21b4cc3494bbe4a13b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:34 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:48122
I20260810 06:37:34.619997 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 2723
I20260810 06:37:34.629419 3746 tablet_replica.cc:330] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd: stopping tablet replica
I20260810 06:37:34.630115 3746 raft_consensus.cc:2229] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:37:34.630775 3746 raft_consensus.cc:2258] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:34.632259 3746 ts_tablet_manager.cc:1905] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:34.634123 3744 tablet_replica.cc:330] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e: stopping tablet replica
I20260810 06:37:34.634366 3744 raft_consensus.cc:2229] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:37:34.634755 3744 raft_consensus.cc:2258] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:34.636018 3742 tablet_replica.cc:330] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30: stopping tablet replica
I20260810 06:37:34.636121 3744 ts_tablet_manager.cc:1905] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:34.636221 3742 raft_consensus.cc:2229] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:37:34.636464 3742 raft_consensus.cc:2258] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:34.637472 3742 ts_tablet_manager.cc:1905] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:34.641187 3746 ts_tablet_manager.cc:1918] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:37:34.641367 3746 log.cc:1196] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/71e2e348391c4b21b4cc3494bbe4a13b
I20260810 06:37:34.641952 3746 ts_tablet_manager.cc:1939] T 71e2e348391c4b21b4cc3494bbe4a13b P c083e39d169f46c7b8e79b96aa62e3dd: Deleting consensus metadata
I20260810 06:37:34.642664 3744 ts_tablet_manager.cc:1918] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:37:34.642814 3744 log.cc:1196] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/71e2e348391c4b21b4cc3494bbe4a13b
I20260810 06:37:34.643185 2548 catalog_manager.cc:4841] TS c083e39d169f46c7b8e79b96aa62e3dd (127.31.59.194:39235): tablet 71e2e348391c4b21b4cc3494bbe4a13b (table test-table [id=e5b7407981ff4be39da2182b2ac5d94c]) successfully deleted
I20260810 06:37:34.643296 3744 ts_tablet_manager.cc:1939] T 71e2e348391c4b21b4cc3494bbe4a13b P 62cc8fd91a0544cebe5825c9d9651c6e: Deleting consensus metadata
I20260810 06:37:34.644351 2549 catalog_manager.cc:4841] TS 62cc8fd91a0544cebe5825c9d9651c6e (127.31.59.195:44383): tablet 71e2e348391c4b21b4cc3494bbe4a13b (table test-table [id=e5b7407981ff4be39da2182b2ac5d94c]) successfully deleted
I20260810 06:37:34.645083 3742 ts_tablet_manager.cc:1918] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:37:34.645193 3742 log.cc:1196] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/71e2e348391c4b21b4cc3494bbe4a13b
I20260810 06:37:34.645551 3742 ts_tablet_manager.cc:1939] T 71e2e348391c4b21b4cc3494bbe4a13b P e4125072468246afa375da1a22c5be30: Deleting consensus metadata
I20260810 06:37:34.646415 2550 catalog_manager.cc:4841] TS e4125072468246afa375da1a22c5be30 (127.31.59.193:35223): tablet 71e2e348391c4b21b4cc3494bbe4a13b (table test-table [id=e5b7407981ff4be39da2182b2ac5d94c]) successfully deleted
Aug 10 06:37:34 dist-test-slave-4tdj krb5kdc[2495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343852, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:37:34.654739 3749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:34.621486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52030 (local address 127.31.59.193:35223)
0810 06:37:34.635283 (+ 13797us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:34.635290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:34.635314 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:37:34.635374 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:34.635379 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:34.635456 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:34.635592 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:34.637270 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:34.638918 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:34.640833 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:34.641502 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:34.641626 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:34.649445 (+ 7819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:34.649460 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:34.649477 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:34.649510 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:34.652229 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:34.652814 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:34.652822 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:34.652825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:34.653039 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:34.653323 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:34.653326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:34.653328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:34.653801 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:34.654106 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:34.654248 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 06:37:34.654460 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13665,"thread_start_us":13575,"threads_started":1}
W20260810 06:37:34.655402 2958 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52030
W20260810 06:37:34.656109 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 2723: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:37:34.656303 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 2723
I20260810 06:37:34.686424 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 2973
Aug 10 06:37:34 dist-test-slave-4tdj krb5kdc[2495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343852, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.194@KRBTEST.COM
I20260810 06:37:34.701103 3750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:34.687212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48132 (local address 127.31.59.194:39235)
0810 06:37:34.687722 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:34.687726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:34.687742 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:37:34.687798 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:34.687802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:34.687860 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:34.687955 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:34.689396 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:34.690120 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:34.691416 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:34.691816 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:34.691915 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:34.696074 (+ 4159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:34.696088 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:34.696104 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:34.696143 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:34.698837 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:34.699428 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:34.699435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:34.699438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:34.699597 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:34.699883 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:34.699886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:34.699888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:34.700233 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:34.700456 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:34.700645 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 06:37:34.700808 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":246,"threads_started":1}
W20260810 06:37:34.701844 3205 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48132
W20260810 06:37:34.702517 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 2973: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:37:34.702678 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 2973
I20260810 06:37:34.727718 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 3215
Aug 10 06:37:34 dist-test-slave-4tdj krb5kdc[2495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343852, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.59.195@KRBTEST.COM
I20260810 06:37:34.742698 3751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:34.728461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43240 (local address 127.31.59.195:44383)
0810 06:37:34.729001 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:34.729005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:34.729021 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:37:34.729075 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:34.729078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:34.729143 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:34.729255 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:34.730622 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:34.731348 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:34.732604 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:34.733056 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:34.733152 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:34.737273 (+ 4121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:34.737288 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:34.737306 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:34.737337 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:34.740236 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:34.740871 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:34.740879 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:34.740881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:34.741007 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:34.741339 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:34.741342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:34.741344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:34.741705 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:34.741931 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:34.742169 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 06:37:34.742409 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":308,"threads_started":1}
W20260810 06:37:34.743458 3457 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43240
W20260810 06:37:34.744089 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 3215: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:37:34.744242 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 3215
I20260810 06:37:34.776590 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 2518
I20260810 06:37:34.777719 2661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:34.891181 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 2518
2026-08-10T06:37:34Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9390 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:37:34 dist-test-slave-4tdj krb5kdc[3758](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:37:34 dist-test-slave-4tdj krb5kdc[3758](info): set up 2 sockets
Aug 10 06:37:34 dist-test-slave-4tdj krb5kdc[3758](info): commencing operation
krb5kdc: starting...
W20260810 06:37:36.100764 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:37:36 dist-test-slave-4tdj krb5kdc[3758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:37:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:37:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:37:36.289822 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:38285
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:44955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:38285
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:36.603348 3778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:36.603627 3778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:36.603761 3778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:36.610703 3778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:37:36.610806 3778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:36.610883 3778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:36.610956 3778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:37:36.611022 3778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:37:36.620464 3778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:44955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:38285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:38285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:36.621644 3778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:36.622876 3778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:36.633971 3786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:36.634063 3787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:36.633992 3791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:36.634624 3778 server_base.cc:1034] running on GCE node
I20260810 06:37:36.635386 3778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:36.636649 3778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:36.637851 3778 hybrid_clock.cc:648] HybridClock initialized: now 1786343856637799 us; error 46 us; skew 500 ppm
I20260810 06:37:36.638145 3778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:36 dist-test-slave-4tdj krb5kdc[3758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343856, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:36.640962 3778 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:37:36.642249 3778 webserver.cc:460] Webserver started at http://127.31.59.254:44887/ using document root <none> and password file <none>
I20260810 06:37:36.642880 3778 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:36.642982 3778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:36.643242 3778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:36.646082 3778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "68c87efc96974c3b96f7a38a13b0ad43"
format_stamp: "Formatted at 2026-08-10 06:37:36 on dist-test-slave-4tdj"
server_key: "50838176765aed3eae5356ac013fd8d2"
server_key_iv: "54e42903f6b0b59f7acd4945b98a3fef"
server_key_version: "encryptionkey@0"
I20260810 06:37:36.646816 3778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "68c87efc96974c3b96f7a38a13b0ad43"
format_stamp: "Formatted at 2026-08-10 06:37:36 on dist-test-slave-4tdj"
server_key: "50838176765aed3eae5356ac013fd8d2"
server_key_iv: "54e42903f6b0b59f7acd4945b98a3fef"
server_key_version: "encryptionkey@0"
I20260810 06:37:36.651597 3778 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260810 06:37:36.654695 3808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:36.655915 3778 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 06:37:36.656055 3778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "68c87efc96974c3b96f7a38a13b0ad43"
format_stamp: "Formatted at 2026-08-10 06:37:36 on dist-test-slave-4tdj"
server_key: "50838176765aed3eae5356ac013fd8d2"
server_key_iv: "54e42903f6b0b59f7acd4945b98a3fef"
server_key_version: "encryptionkey@0"
I20260810 06:37:36.656224 3778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:36.674412 3778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:36.677958 3778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:36.678171 3778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:36.694648 3778 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:38285
I20260810 06:37:36.694665 3888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:38285 every 8 connection(s)
I20260810 06:37:36.695945 3778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:37:36.699460 3891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:36.705894 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 3778
I20260810 06:37:36.706075 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:37:36.706774 31983 external_mini_cluster.cc:1428] Setting key 7aa9ab5c5c70c71484797c862b15f2f8
I20260810 06:37:36.709465 3891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43: Bootstrap starting.
I20260810 06:37:36.712782 3891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:36.713984 3891 log.cc:824] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:36.717028 3891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43: No bootstrap required, opened a new log
Aug 10 06:37:36 dist-test-slave-4tdj krb5kdc[3758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:36.725668 3891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c87efc96974c3b96f7a38a13b0ad43" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38285 } }
I20260810 06:37:36.726127 3891 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:36.726236 3891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c87efc96974c3b96f7a38a13b0ad43, State: Initialized, Role: FOLLOWER
I20260810 06:37:36.726886 3891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c87efc96974c3b96f7a38a13b0ad43" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38285 } }
I20260810 06:37:36.727133 3891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:37:36.727268 3891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:37:36.727432 3891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:36.729447 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:36.708635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51286 (local address 127.31.59.254:38285)
0810 06:37:36.709409 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:36.709419 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:36.709461 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:37:36.711164 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:36.711193 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:36.711547 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:36.711834 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:36.713646 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:36.715337 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:36.717001 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:36.717684 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:36.717809 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:36.722337 (+ 4528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:36.722346 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:36.722361 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:36.722401 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:36.725311 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:36.726086 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:36.726094 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:36.726103 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:36.726331 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:36.726748 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:36.726751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:36.726753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:36.727333 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:36.727608 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:36.728060 (+ 452us) server_negotiation.cc:299] Negotiation successful
0810 06:37:36.728482 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":329,"threads_started":1}
I20260810 06:37:36.729817 3891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c87efc96974c3b96f7a38a13b0ad43" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38285 } }
I20260810 06:37:36.730356 3891 leader_election.cc:302] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68c87efc96974c3b96f7a38a13b0ad43; no voters:
I20260810 06:37:36.730988 3891 leader_election.cc:288] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:37:36.731107 3899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:36.732120 3899 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43 [term 1 LEADER]: Becoming Leader. State: Replica: 68c87efc96974c3b96f7a38a13b0ad43, State: Running, Role: LEADER
I20260810 06:37:36.732676 3899 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c87efc96974c3b96f7a38a13b0ad43" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38285 } }
I20260810 06:37:36.733211 3891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:37:36.735509 3901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68c87efc96974c3b96f7a38a13b0ad43. Latest consensus state: current_term: 1 leader_uuid: "68c87efc96974c3b96f7a38a13b0ad43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c87efc96974c3b96f7a38a13b0ad43" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38285 } } }
I20260810 06:37:36.735584 3900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68c87efc96974c3b96f7a38a13b0ad43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c87efc96974c3b96f7a38a13b0ad43" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38285 } } }
I20260810 06:37:36.735997 3900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:36.736020 3901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:36.742810 3908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:37:36.747184 3908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:37:36.755918 3908 catalog_manager.cc:1347] Generated new cluster ID: 744c18bab3a94643bf755e3562827a34
I20260810 06:37:36.756027 3908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:37:36.790874 3908 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:37:36.792176 3908 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:37:36.799283 3908 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68c87efc96974c3b96f7a38a13b0ad43: Generated new TSK 0
I20260810 06:37:36.800094 3908 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
I20260810 06:37:36.858016 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:38285
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:44955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:37.183544 3935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:37.183816 3935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:37.183926 3935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:37.191107 3935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:37.191222 3935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:37.191350 3935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:37:37.200897 3935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:44955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:38285
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:37.202122 3935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:37.203437 3935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:37.218835 3949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:37.218762 3954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:37.218803 3951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:37.219703 3935 server_base.cc:1034] running on GCE node
I20260810 06:37:37.220335 3935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:37.221267 3935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:37.222471 3935 hybrid_clock.cc:648] HybridClock initialized: now 1786343857222400 us; error 66 us; skew 500 ppm
I20260810 06:37:37.222781 3935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:37 dist-test-slave-4tdj krb5kdc[3758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343857, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:37.225912 3935 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:37:37.227321 3935 webserver.cc:460] Webserver started at http://127.31.59.193:44787/ using document root <none> and password file <none>
I20260810 06:37:37.227965 3935 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:37.228114 3935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:37.228422 3935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:37.231402 3935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "583bb189f7ba42c88c81f61ada433723"
format_stamp: "Formatted at 2026-08-10 06:37:37 on dist-test-slave-4tdj"
server_key: "4eeeec957ec1f7016819eceddad060aa"
server_key_iv: "7b69afa2e7f69ee080f1722f0191fe32"
server_key_version: "encryptionkey@0"
I20260810 06:37:37.232154 3935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "583bb189f7ba42c88c81f61ada433723"
format_stamp: "Formatted at 2026-08-10 06:37:37 on dist-test-slave-4tdj"
server_key: "4eeeec957ec1f7016819eceddad060aa"
server_key_iv: "7b69afa2e7f69ee080f1722f0191fe32"
server_key_version: "encryptionkey@0"
I20260810 06:37:37.237062 3935 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 06:37:37.240381 3967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:37.241647 3935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:37:37.241816 3935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "583bb189f7ba42c88c81f61ada433723"
format_stamp: "Formatted at 2026-08-10 06:37:37 on dist-test-slave-4tdj"
server_key: "4eeeec957ec1f7016819eceddad060aa"
server_key_iv: "7b69afa2e7f69ee080f1722f0191fe32"
server_key_version: "encryptionkey@0"
I20260810 06:37:37.241991 3935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:37.258500 3935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:37.262257 3935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:37.262533 3935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:37.263432 3935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:37.264775 3935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:37.264858 3935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:37.264973 3935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:37.265035 3935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:37.295378 3935 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:37571
I20260810 06:37:37.295419 4157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:37571 every 8 connection(s)
I20260810 06:37:37.296566 3935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:37:37.305431 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 3935
I20260810 06:37:37.305619 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:37:37.306238 31983 external_mini_cluster.cc:1428] Setting key 64c4c6bf54ebdd2b4233c6c7f0fa4a80
Aug 10 06:37:37 dist-test-slave-4tdj krb5kdc[3758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343857, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:37.315804 4160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:37.299022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:45937 (local address 127.31.59.254:38285)
0810 06:37:37.299447 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:37.299452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:37.300358 (+ 906us) server_negotiation.cc:407] Connection header received
0810 06:37:37.302218 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:37.302224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:37.302332 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:37.302427 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:37.303899 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:37.304627 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:37.306245 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:37.306696 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:37.306812 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:37.310345 (+ 3533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:37.310353 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:37.310358 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:37.310396 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:37.312829 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:37.313527 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:37.313532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:37.313534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:37.313653 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:37.314073 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:37.314079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:37.314082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:37.314383 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:37.314520 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:37.315205 (+ 685us) server_negotiation.cc:299] Negotiation successful
0810 06:37:37.315427 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":203,"threads_started":1}
I20260810 06:37:37.316742 4163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:37.299502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:38285 (local address 127.31.59.193:45937)
0810 06:37:37.300197 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:37.300237 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:37.301937 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:37.302685 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:37.302694 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:37.303195 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:37.303735 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:37.303753 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:37.304794 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:37.304798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:37.306108 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:37.306121 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:37.306924 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:37.306931 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:37.307152 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:37.307818 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:37.307855 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:37.310106 (+ 2251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:37.312964 (+ 2858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:37.312973 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:37.312999 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:37.313407 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:37.313768 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:37.313773 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:37.313777 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:37.313962 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:37.314510 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:37.314517 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:37.314843 (+ 326us) client_negotiation.cc:769] Sending connection context
0810 06:37:37.315097 (+ 254us) client_negotiation.cc:240] Negotiation successful
0810 06:37:37.315619 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":254,"threads_started":1}
I20260810 06:37:37.318282 4158 heartbeater.cc:347] Connected to a master server at 127.31.59.254:38285
I20260810 06:37:37.318596 4158 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:37.319278 4158 heartbeater.cc:511] Master 127.31.59.254:38285 requested a full tablet report, sending...
I20260810 06:37:37.321602 3829 ts_manager.cc:194] Registered new tserver with Master: 583bb189f7ba42c88c81f61ada433723 (127.31.59.193:37571)
I20260810 06:37:37.323665 3829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:45937
WARNING: no policy specified for kudu/127.31.59.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
I20260810 06:37:37.350940 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:38285
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:44955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:37.667725 4169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:37.667986 4169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:37.668123 4169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:37.675195 4169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:37.675297 4169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:37.675446 4169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:37:37.684626 4169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:44955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:38285
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:37.685794 4169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:37.687058 4169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:37.699668 4176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:37.699628 4178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:37.700297 4182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:37.700843 4169 server_base.cc:1034] running on GCE node
I20260810 06:37:37.701386 4169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:37.702234 4169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:37.703459 4169 hybrid_clock.cc:648] HybridClock initialized: now 1786343857703400 us; error 47 us; skew 500 ppm
I20260810 06:37:37.703745 4169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:37 dist-test-slave-4tdj krb5kdc[3758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343857, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:37.706676 4169 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:37:37.708032 4169 webserver.cc:460] Webserver started at http://127.31.59.194:39043/ using document root <none> and password file <none>
I20260810 06:37:37.708652 4169 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:37.708753 4169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:37.709038 4169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:37.711737 4169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "534db29c60154bacbf5082a391e20fc7"
format_stamp: "Formatted at 2026-08-10 06:37:37 on dist-test-slave-4tdj"
server_key: "fe4a532f4c8d850e281bd662899cb1d2"
server_key_iv: "3172a5df4d5f0d6fd8e8a732671045bc"
server_key_version: "encryptionkey@0"
I20260810 06:37:37.712466 4169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "534db29c60154bacbf5082a391e20fc7"
format_stamp: "Formatted at 2026-08-10 06:37:37 on dist-test-slave-4tdj"
server_key: "fe4a532f4c8d850e281bd662899cb1d2"
server_key_iv: "3172a5df4d5f0d6fd8e8a732671045bc"
server_key_version: "encryptionkey@0"
I20260810 06:37:37.717409 4169 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:37:37.720846 4190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:37.722067 4169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:37:37.722213 4169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "534db29c60154bacbf5082a391e20fc7"
format_stamp: "Formatted at 2026-08-10 06:37:37 on dist-test-slave-4tdj"
server_key: "fe4a532f4c8d850e281bd662899cb1d2"
server_key_iv: "3172a5df4d5f0d6fd8e8a732671045bc"
server_key_version: "encryptionkey@0"
I20260810 06:37:37.722445 4169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:37.746762 4169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:37.750829 4169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:37.751092 4169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:37.752074 4169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:37.753316 4169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:37.753387 4169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:37.753500 4169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:37.753553 4169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:37.783018 4169 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:38513
I20260810 06:37:37.783071 4376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:38513 every 8 connection(s)
I20260810 06:37:37.784210 4169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:37:37.789023 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 4169
I20260810 06:37:37.789170 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:37:37.789580 31983 external_mini_cluster.cc:1428] Setting key d460790566a7af240231fc48a3b69bf8
Aug 10 06:37:37 dist-test-slave-4tdj krb5kdc[3758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343857, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:37.803344 4160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:37.786650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:59941 (local address 127.31.59.254:38285)
0810 06:37:37.786856 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:37.786861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:37.787888 (+ 1027us) server_negotiation.cc:407] Connection header received
0810 06:37:37.789719 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:37.789725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:37.789808 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:37.789899 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:37.791558 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:37.792287 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:37.793764 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:37.794170 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:37.794280 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:37.797607 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:37.797612 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:37.797615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:37.797647 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:37.800080 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:37.800905 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:37.800908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:37.800911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:37.801045 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:37.801551 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:37.801555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:37.801557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:37.801851 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:37.801994 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:37.802867 (+ 873us) server_negotiation.cc:299] Negotiation successful
0810 06:37:37.803077 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:37:37.804556 4381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:37.787067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:38285 (local address 127.31.59.194:59941)
0810 06:37:37.787734 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:37.787776 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:37.789468 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:37.790158 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:37.790167 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:37.790787 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:37.791406 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:37.791426 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:37.792438 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:37.792442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:37.793623 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:37.793635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:37.794405 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:37.794413 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:37.794627 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:37.795223 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:37.795244 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:37.797406 (+ 2162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:37.800212 (+ 2806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:37.800221 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:37.800236 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:37.800802 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:37.801148 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:37.801152 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:37.801156 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:37.801392 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:37.801994 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:37.802002 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:37.802437 (+ 435us) client_negotiation.cc:769] Sending connection context
0810 06:37:37.802746 (+ 309us) client_negotiation.cc:240] Negotiation successful
0810 06:37:37.803224 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":250,"threads_started":1}
I20260810 06:37:37.806083 4378 heartbeater.cc:347] Connected to a master server at 127.31.59.254:38285
I20260810 06:37:37.806416 4378 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:37.807152 4378 heartbeater.cc:511] Master 127.31.59.254:38285 requested a full tablet report, sending...
I20260810 06:37:37.808679 3829 ts_manager.cc:194] Registered new tserver with Master: 534db29c60154bacbf5082a391e20fc7 (127.31.59.194:38513)
WARNING: no policy specified for kudu/127.31.59.195@KRBTEST.COM; defaulting to no policy
I20260810 06:37:37.809778 3829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:59941
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
I20260810 06:37:37.833163 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:38285
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:44955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:38.144062 4386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:38.144336 4386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:38.144428 4386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:38.151533 4386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:38.151639 4386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:38.151763 4386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:37:38.161230 4386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:44955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:38285
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:38.162453 4386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:38.163740 4386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:38.175868 4396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:38.175989 4402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:38.176152 4404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:38.176851 4386 server_base.cc:1034] running on GCE node
I20260810 06:37:38.177379 4386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:38.178194 4386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:38.179404 4386 hybrid_clock.cc:648] HybridClock initialized: now 1786343858179339 us; error 49 us; skew 500 ppm
I20260810 06:37:38.179713 4386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:38 dist-test-slave-4tdj krb5kdc[3758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343858, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:38.182785 4386 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:37:38.184134 4386 webserver.cc:460] Webserver started at http://127.31.59.195:42943/ using document root <none> and password file <none>
I20260810 06:37:38.184773 4386 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:38.184885 4386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:38.185173 4386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:38.187935 4386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "6e126f7664c84035aeb057ea4bad5915"
format_stamp: "Formatted at 2026-08-10 06:37:38 on dist-test-slave-4tdj"
server_key: "257d774e830a08ae82ea0f118c1da676"
server_key_iv: "c17f6b4f56384511a3a97f0ff0b0cbf4"
server_key_version: "encryptionkey@0"
I20260810 06:37:38.188690 4386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "6e126f7664c84035aeb057ea4bad5915"
format_stamp: "Formatted at 2026-08-10 06:37:38 on dist-test-slave-4tdj"
server_key: "257d774e830a08ae82ea0f118c1da676"
server_key_iv: "c17f6b4f56384511a3a97f0ff0b0cbf4"
server_key_version: "encryptionkey@0"
I20260810 06:37:38.193476 4386 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:37:38.196717 4418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:38.197899 4386 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 06:37:38.198047 4386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "6e126f7664c84035aeb057ea4bad5915"
format_stamp: "Formatted at 2026-08-10 06:37:38 on dist-test-slave-4tdj"
server_key: "257d774e830a08ae82ea0f118c1da676"
server_key_iv: "c17f6b4f56384511a3a97f0ff0b0cbf4"
server_key_version: "encryptionkey@0"
I20260810 06:37:38.198226 4386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:38.208374 4386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:38.212262 4386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:38.212560 4386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:38.213441 4386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:38.214819 4386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:38.214893 4386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:38.214972 4386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:38.215013 4386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:38.244905 4386 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:43753
I20260810 06:37:38.244949 4607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:43753 every 8 connection(s)
I20260810 06:37:38.246169 4386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:37:38.250759 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 4386
I20260810 06:37:38.250910 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:37:38.251322 31983 external_mini_cluster.cc:1428] Setting key 0f575d64a9202284a8c0253ba6378c5c
Aug 10 06:37:38 dist-test-slave-4tdj krb5kdc[3758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343858, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:38.264469 4160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:38.248558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:45783 (local address 127.31.59.254:38285)
0810 06:37:38.248723 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:38.248727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:38.249851 (+ 1124us) server_negotiation.cc:407] Connection header received
0810 06:37:38.251725 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:38.251731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:38.251831 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:38.251939 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:38.253725 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:38.254655 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:38.256081 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:38.256480 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:38.256580 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:38.259604 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:38.259609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:38.259612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:38.259667 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:38.262154 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:38.262776 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:38.262780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:38.262782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:38.262891 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:38.263210 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:38.263213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:38.263215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:38.263458 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:38.263566 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:38.264086 (+ 520us) server_negotiation.cc:299] Negotiation successful
0810 06:37:38.264230 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:37:38.265448 4612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:38.249008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:38285 (local address 127.31.59.195:45783)
0810 06:37:38.249697 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:38.249739 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:38.251435 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:38.252214 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:38.252226 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:38.252892 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:38.253558 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:38.253575 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:38.254773 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:38.254777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:38.255947 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:38.255963 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:38.256646 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:38.256651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:38.256829 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:38.257390 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:38.257410 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:38.259409 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:38.262265 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:38.262271 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:38.262281 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:38.262683 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:38.262962 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:38.262965 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:38.262967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:38.263125 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:38.263547 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:38.263552 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:38.263813 (+ 261us) client_negotiation.cc:769] Sending connection context
0810 06:37:38.264018 (+ 205us) client_negotiation.cc:240] Negotiation successful
0810 06:37:38.264356 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":258,"threads_started":1}
I20260810 06:37:38.266655 4608 heartbeater.cc:347] Connected to a master server at 127.31.59.254:38285
I20260810 06:37:38.266918 4608 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:38.267542 4608 heartbeater.cc:511] Master 127.31.59.254:38285 requested a full tablet report, sending...
I20260810 06:37:38.268821 3827 ts_manager.cc:194] Registered new tserver with Master: 6e126f7664c84035aeb057ea4bad5915 (127.31.59.195:43753)
I20260810 06:37:38.269623 3827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:45783
I20260810 06:37:38.278234 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 06:37:38.309037 4160 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:37:38.306685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.31.59.254:38285)
0810 06:37:38.306899 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:38.306906 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:38.307318 (+ 412us) server_negotiation.cc:407] Connection header received
0810 06:37:38.307503 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:38.307508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:38.307582 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:38.307665 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:38.308781 (+ 1116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48604: BlockingRecv error: recv got EOF from 127.0.0.1:48604 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:37:38.312071 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 3935
W20260810 06:37:38.314747 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 3935: Not authorized: Client connection negotiation failed: client connection to 127.31.59.193:37571: server requires authentication, but client does not have Kerberos credentials available
I20260810 06:37:38.314947 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 3935
I20260810 06:37:38.338260 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 4169
W20260810 06:37:38.340768 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 4169: Not authorized: Client connection negotiation failed: client connection to 127.31.59.194:38513: server requires authentication, but client does not have Kerberos credentials available
I20260810 06:37:38.340895 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 4169
I20260810 06:37:38.358867 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 4386
W20260810 06:37:38.361613 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 4386: Not authorized: Client connection negotiation failed: client connection to 127.31.59.195:43753: server requires authentication, but client does not have Kerberos credentials available
I20260810 06:37:38.361755 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 4386
I20260810 06:37:38.378566 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 3778
I20260810 06:37:38.379601 3883 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:38.482519 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 3778
2026-08-10T06:37:38Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3589 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:37:38 dist-test-slave-4tdj krb5kdc[4633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:37:38 dist-test-slave-4tdj krb5kdc[4633](info): set up 2 sockets
Aug 10 06:37:38 dist-test-slave-4tdj krb5kdc[4633](info): commencing operation
krb5kdc: starting...
W20260810 06:37:39.679157 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:37:39 dist-test-slave-4tdj krb5kdc[4633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:37:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:37:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:37:39.865430 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:37175
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:42199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:37175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:40.179783 4652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:40.180051 4652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:40.180189 4652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:40.187170 4652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:37:40.187265 4652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:40.187335 4652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:40.187405 4652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:37:40.187461 4652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:37:40.196817 4652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:42199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:37175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:37175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:40.198010 4652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:40.199303 4652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:40.210455 4663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:40.210534 4664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:40.210553 4666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:40.211090 4652 server_base.cc:1034] running on GCE node
I20260810 06:37:40.211825 4652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:40.213162 4652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:40.214387 4652 hybrid_clock.cc:648] HybridClock initialized: now 1786343860214346 us; error 42 us; skew 500 ppm
I20260810 06:37:40.214725 4652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:40 dist-test-slave-4tdj krb5kdc[4633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343860, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:40.221895 4652 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:37:40.223577 4652 webserver.cc:460] Webserver started at http://127.31.59.254:40071/ using document root <none> and password file <none>
I20260810 06:37:40.224293 4652 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:40.224426 4652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:40.224750 4652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:40.228027 4652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "4d3e3e4d44e34a38b9490f9fc86f1dcf"
format_stamp: "Formatted at 2026-08-10 06:37:40 on dist-test-slave-4tdj"
server_key: "4e6693505f5d0cf81e220fa224dfa710"
server_key_iv: "cc75825b5b9436d59d8c7c857c15ac02"
server_key_version: "encryptionkey@0"
I20260810 06:37:40.228863 4652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "4d3e3e4d44e34a38b9490f9fc86f1dcf"
format_stamp: "Formatted at 2026-08-10 06:37:40 on dist-test-slave-4tdj"
server_key: "4e6693505f5d0cf81e220fa224dfa710"
server_key_iv: "cc75825b5b9436d59d8c7c857c15ac02"
server_key_version: "encryptionkey@0"
I20260810 06:37:40.234386 4652 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260810 06:37:40.237833 4678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:40.239193 4652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:37:40.239358 4652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "4d3e3e4d44e34a38b9490f9fc86f1dcf"
format_stamp: "Formatted at 2026-08-10 06:37:40 on dist-test-slave-4tdj"
server_key: "4e6693505f5d0cf81e220fa224dfa710"
server_key_iv: "cc75825b5b9436d59d8c7c857c15ac02"
server_key_version: "encryptionkey@0"
I20260810 06:37:40.239553 4652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:40.266412 4652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:40.270478 4652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:40.270771 4652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:40.289260 4652 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:37175
I20260810 06:37:40.289289 4777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:37175 every 8 connection(s)
I20260810 06:37:40.290691 4652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:37:40.292042 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 4652
I20260810 06:37:40.292296 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:37:40.293052 31983 external_mini_cluster.cc:1428] Setting key 644cb97a757726d2340825880ef58d3a
I20260810 06:37:40.295323 4778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:37:40 dist-test-slave-4tdj krb5kdc[4633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:40.308233 4778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf: Bootstrap starting.
I20260810 06:37:40.312156 4778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:40.313442 4778 log.cc:824] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:40.316819 4778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf: No bootstrap required, opened a new log
I20260810 06:37:40.317339 4781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:40.294991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36900 (local address 127.31.59.254:37175)
0810 06:37:40.295823 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:40.295834 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:40.295878 (+ 44us) server_negotiation.cc:407] Connection header received
0810 06:37:40.297610 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:40.297632 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:40.298036 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:40.298425 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:40.300289 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:40.301749 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:40.303450 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:40.304154 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:40.304297 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:40.309022 (+ 4725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:40.309032 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:40.309049 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:40.309091 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:40.312914 (+ 3823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:40.313643 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:40.313651 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:40.313660 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:40.313867 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:40.314284 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:40.314287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:40.314310 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:40.314935 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:40.315225 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:40.315623 (+ 398us) server_negotiation.cc:299] Negotiation successful
0810 06:37:40.316047 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":405,"threads_started":1}
I20260810 06:37:40.321024 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:37:40.326140 4778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d3e3e4d44e34a38b9490f9fc86f1dcf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 37175 } }
I20260810 06:37:40.326613 4778 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:40.326717 4778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d3e3e4d44e34a38b9490f9fc86f1dcf, State: Initialized, Role: FOLLOWER
I20260810 06:37:40.327346 4778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d3e3e4d44e34a38b9490f9fc86f1dcf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 37175 } }
I20260810 06:37:40.327688 4778 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:37:40.327817 4778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:37:40.327975 4778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:40.330726 4778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d3e3e4d44e34a38b9490f9fc86f1dcf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 37175 } }
I20260810 06:37:40.331209 4778 leader_election.cc:302] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d3e3e4d44e34a38b9490f9fc86f1dcf; no voters:
I20260810 06:37:40.331816 4778 leader_election.cc:288] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:37:40.331921 4786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:40.334050 4778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:37:40.338434 4786 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf [term 1 LEADER]: Becoming Leader. State: Replica: 4d3e3e4d44e34a38b9490f9fc86f1dcf, State: Running, Role: LEADER
I20260810 06:37:40.339120 4786 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d3e3e4d44e34a38b9490f9fc86f1dcf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 37175 } }
I20260810 06:37:40.344735 4788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d3e3e4d44e34a38b9490f9fc86f1dcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d3e3e4d44e34a38b9490f9fc86f1dcf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 37175 } } }
I20260810 06:37:40.345144 4788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:40.345634 4790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d3e3e4d44e34a38b9490f9fc86f1dcf. Latest consensus state: current_term: 1 leader_uuid: "4d3e3e4d44e34a38b9490f9fc86f1dcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d3e3e4d44e34a38b9490f9fc86f1dcf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 37175 } } }
I20260810 06:37:40.345922 4790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:40.347756 4805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:37:40.352869 4805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:37:40.362495 4805 catalog_manager.cc:1347] Generated new cluster ID: 88c1b8635b574b409d6820f68d97ca15
I20260810 06:37:40.362609 4805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:37:40.375022 4781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:40.364548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36914 (local address 127.31.59.254:37175)
0810 06:37:40.364706 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:40.364712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:40.366867 (+ 2155us) server_negotiation.cc:407] Connection header received
0810 06:37:40.367043 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:40.367050 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:40.367134 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:40.367242 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:40.368399 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:40.369583 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:40.371341 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:40.372019 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:40.372146 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:40.373219 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:40.373226 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:40.373230 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:40.374180 (+ 950us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:40.374242 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:40.374270 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:40.374410 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:40.374529 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 06:37:40.374628 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:37:40.389468 4805 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:37:40.390762 4805 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:37:40.395973 4805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d3e3e4d44e34a38b9490f9fc86f1dcf: Generated new TSK 0
I20260810 06:37:40.396916 4805 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:37:40.407261 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 4652
I20260810 06:37:40.408450 4770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:40.497928 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 4652
2026-08-10T06:37:40Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2013 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:37:40 dist-test-slave-4tdj krb5kdc[4822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:37:40 dist-test-slave-4tdj krb5kdc[4822](info): set up 2 sockets
Aug 10 06:37:40 dist-test-slave-4tdj krb5kdc[4822](info): commencing operation
krb5kdc: starting...
W20260810 06:37:41.704063 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:37:41 dist-test-slave-4tdj krb5kdc[4822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:37:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:37:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:37:41.888713 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40989
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:40997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:40989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:42.206821 4844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:42.207156 4844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:42.207360 4844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:42.214813 4844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:37:42.214936 4844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:42.214987 4844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:42.215045 4844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:37:42.215090 4844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:37:42.224753 4844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:40997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:40989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:42.225981 4844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:42.227275 4844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:42.238667 4857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:42.238868 4856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:42.238803 4859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:42.239848 4844 server_base.cc:1034] running on GCE node
I20260810 06:37:42.240675 4844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:42.242054 4844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:42.243266 4844 hybrid_clock.cc:648] HybridClock initialized: now 1786343862243195 us; error 57 us; skew 500 ppm
I20260810 06:37:42.243669 4844 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:42 dist-test-slave-4tdj krb5kdc[4822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343862, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:42.246800 4844 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:37:42.248214 4844 webserver.cc:460] Webserver started at http://127.31.59.254:36799/ using document root <none> and password file <none>
I20260810 06:37:42.248869 4844 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:42.248998 4844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:42.249310 4844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:42.252347 4844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "d18637e1861e4ef99af756c78ded6c05"
format_stamp: "Formatted at 2026-08-10 06:37:42 on dist-test-slave-4tdj"
server_key: "412ee04c257b1a49b52b405547855e94"
server_key_iv: "8a2fe993a17d703b76bebb5b1b033b47"
server_key_version: "encryptionkey@0"
I20260810 06:37:42.253149 4844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "d18637e1861e4ef99af756c78ded6c05"
format_stamp: "Formatted at 2026-08-10 06:37:42 on dist-test-slave-4tdj"
server_key: "412ee04c257b1a49b52b405547855e94"
server_key_iv: "8a2fe993a17d703b76bebb5b1b033b47"
server_key_version: "encryptionkey@0"
I20260810 06:37:42.258278 4844 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 06:37:42.261744 4871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:42.263074 4844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 06:37:42.263250 4844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "d18637e1861e4ef99af756c78ded6c05"
format_stamp: "Formatted at 2026-08-10 06:37:42 on dist-test-slave-4tdj"
server_key: "412ee04c257b1a49b52b405547855e94"
server_key_iv: "8a2fe993a17d703b76bebb5b1b033b47"
server_key_version: "encryptionkey@0"
I20260810 06:37:42.263451 4844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:42.277844 4844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:42.281525 4844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:42.281800 4844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:42.299377 4844 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:40989
I20260810 06:37:42.299404 4962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:40989 every 8 connection(s)
I20260810 06:37:42.300741 4844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:37:42.304184 4967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:42.304862 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 4844
I20260810 06:37:42.305109 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:37:42.305920 31983 external_mini_cluster.cc:1428] Setting key 6b04ca660f5130639f016a7f6daf74be
I20260810 06:37:42.317561 4967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05: Bootstrap starting.
Aug 10 06:37:42 dist-test-slave-4tdj krb5kdc[4822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:42.321008 4967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:42.322073 4967 log.cc:824] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:42.324922 4967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05: No bootstrap required, opened a new log
I20260810 06:37:42.328890 4972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:42.307925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43774 (local address 127.31.59.254:40989)
0810 06:37:42.308649 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:42.308659 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:42.308701 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:37:42.310283 (+ 1582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:42.310332 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:42.310718 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:42.311021 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:42.312635 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:42.313956 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:42.315697 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:42.316333 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:42.316455 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:42.321336 (+ 4881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:42.321348 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:42.321362 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:42.321410 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:42.324657 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:42.325432 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:42.325441 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:42.325450 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:42.325634 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:42.326029 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:42.326035 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:42.326038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:42.326641 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:42.327006 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:42.327443 (+ 437us) server_negotiation.cc:299] Negotiation successful
0810 06:37:42.327856 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":319,"threads_started":1}
I20260810 06:37:42.332695 4967 raft_consensus.cc:348] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18637e1861e4ef99af756c78ded6c05" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40989 } }
I20260810 06:37:42.333019 4967 raft_consensus.cc:374] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:42.333113 4967 raft_consensus.cc:729] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d18637e1861e4ef99af756c78ded6c05, State: Initialized, Role: FOLLOWER
I20260810 06:37:42.333624 4967 consensus_queue.cc:260] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18637e1861e4ef99af756c78ded6c05" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40989 } }
I20260810 06:37:42.333848 4967 raft_consensus.cc:388] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:37:42.333998 4967 raft_consensus.cc:482] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:37:42.334172 4967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:42.336184 4967 raft_consensus.cc:504] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18637e1861e4ef99af756c78ded6c05" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40989 } }
I20260810 06:37:42.336638 4967 leader_election.cc:302] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d18637e1861e4ef99af756c78ded6c05; no voters:
I20260810 06:37:42.337206 4967 leader_election.cc:288] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:37:42.337316 4977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:42.337930 4977 raft_consensus.cc:686] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05 [term 1 LEADER]: Becoming Leader. State: Replica: d18637e1861e4ef99af756c78ded6c05, State: Running, Role: LEADER
I20260810 06:37:42.338438 4977 consensus_queue.cc:237] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18637e1861e4ef99af756c78ded6c05" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40989 } }
I20260810 06:37:42.339087 4967 sys_catalog.cc:564] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:37:42.340876 4982 sys_catalog.cc:455] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d18637e1861e4ef99af756c78ded6c05. Latest consensus state: current_term: 1 leader_uuid: "d18637e1861e4ef99af756c78ded6c05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18637e1861e4ef99af756c78ded6c05" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40989 } } }
I20260810 06:37:42.341250 4982 sys_catalog.cc:458] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:42.342173 4998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:37:42.342015 4978 sys_catalog.cc:455] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d18637e1861e4ef99af756c78ded6c05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18637e1861e4ef99af756c78ded6c05" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40989 } } }
I20260810 06:37:42.342353 4978 sys_catalog.cc:458] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:42.347848 4998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:37:42.355888 4998 catalog_manager.cc:1347] Generated new cluster ID: 64362f381dd2492181f66089fed40570
I20260810 06:37:42.355984 4998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:37:42.364346 4998 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:37:42.365531 4998 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:37:42.377462 4998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d18637e1861e4ef99af756c78ded6c05: Generated new TSK 0
I20260810 06:37:42.378222 4998 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
I20260810 06:37:42.431171 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40989
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:40997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:42.776894 5022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:42.777156 5022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:42.777274 5022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:42.784307 5022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:42.784404 5022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:42.784556 5022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:37:42.793886 5022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:40997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:42.795078 5022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:42.796306 5022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:42.808364 5035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:42.808398 5037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:42.808522 5041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:42.809460 5022 server_base.cc:1034] running on GCE node
I20260810 06:37:42.810042 5022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:42.810949 5022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:42.812157 5022 hybrid_clock.cc:648] HybridClock initialized: now 1786343862812102 us; error 43 us; skew 500 ppm
I20260810 06:37:42.812439 5022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:42 dist-test-slave-4tdj krb5kdc[4822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343862, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:42.815490 5022 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:37:42.816834 5022 webserver.cc:460] Webserver started at http://127.31.59.193:36111/ using document root <none> and password file <none>
I20260810 06:37:42.817433 5022 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:42.817551 5022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:42.817836 5022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:42.820590 5022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "f053408a8c174033bf4ad438bb4fb83b"
format_stamp: "Formatted at 2026-08-10 06:37:42 on dist-test-slave-4tdj"
server_key: "c61f6f3fff24312772d3925a7a584465"
server_key_iv: "dba315abd85dcc57e9e334ce18c2566e"
server_key_version: "encryptionkey@0"
I20260810 06:37:42.821318 5022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "f053408a8c174033bf4ad438bb4fb83b"
format_stamp: "Formatted at 2026-08-10 06:37:42 on dist-test-slave-4tdj"
server_key: "c61f6f3fff24312772d3925a7a584465"
server_key_iv: "dba315abd85dcc57e9e334ce18c2566e"
server_key_version: "encryptionkey@0"
I20260810 06:37:42.826208 5022 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 06:37:42.829532 5051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:42.830958 5022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:37:42.831138 5022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "f053408a8c174033bf4ad438bb4fb83b"
format_stamp: "Formatted at 2026-08-10 06:37:42 on dist-test-slave-4tdj"
server_key: "c61f6f3fff24312772d3925a7a584465"
server_key_iv: "dba315abd85dcc57e9e334ce18c2566e"
server_key_version: "encryptionkey@0"
I20260810 06:37:42.831365 5022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:42.843811 5022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:42.847421 5022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:42.847653 5022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:42.848474 5022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:42.849725 5022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:42.849794 5022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:42.849880 5022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:42.849924 5022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:42.877630 5022 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:43741
I20260810 06:37:42.877667 5249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:43741 every 8 connection(s)
I20260810 06:37:42.878962 5022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:37:42.881897 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 5022
I20260810 06:37:42.882140 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:37:42.882963 31983 external_mini_cluster.cc:1428] Setting key ec354515d50e1b0d58f9b87050726e4f
Aug 10 06:37:42 dist-test-slave-4tdj krb5kdc[4822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343862, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:42.899840 5254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:42.881631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:39297 (local address 127.31.59.254:40989)
0810 06:37:42.882155 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:42.882162 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:42.883256 (+ 1094us) server_negotiation.cc:407] Connection header received
0810 06:37:42.885115 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:42.885120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:42.885181 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:42.885278 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:42.887296 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:42.888051 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:42.890126 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:42.890598 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:42.890697 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:42.894480 (+ 3783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:42.894485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:42.894488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:42.894529 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:42.896873 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:42.897568 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:42.897572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:42.897575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:42.897690 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:42.898106 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:42.898109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:42.898111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:42.898405 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:42.898547 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:42.899242 (+ 695us) server_negotiation.cc:299] Negotiation successful
0810 06:37:42.899462 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":245,"threads_started":1}
I20260810 06:37:42.900758 5255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:42.882157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40989 (local address 127.31.59.193:39297)
0810 06:37:42.883059 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:42.883120 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:42.884842 (+ 1722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:42.885547 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:42.885558 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:42.886264 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:42.887101 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:42.887125 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:42.888210 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:42.888215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:42.889970 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:42.889985 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:42.890815 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:42.890822 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:42.891093 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:42.891767 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:42.891794 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:42.894252 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:42.897009 (+ 2757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:42.897018 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:42.897044 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:42.897438 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:42.897791 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:42.897796 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:42.897801 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:42.897985 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:42.898525 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:42.898533 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:42.898857 (+ 324us) client_negotiation.cc:769] Sending connection context
0810 06:37:42.899106 (+ 249us) client_negotiation.cc:240] Negotiation successful
0810 06:37:42.899621 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":333,"threads_started":1}
I20260810 06:37:42.902383 5250 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40989
I20260810 06:37:42.902695 5250 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.31.59.194@KRBTEST.COM; defaulting to no policy
I20260810 06:37:42.903425 5250 heartbeater.cc:511] Master 127.31.59.254:40989 requested a full tablet report, sending...
I20260810 06:37:42.905546 4897 ts_manager.cc:194] Registered new tserver with Master: f053408a8c174033bf4ad438bb4fb83b (127.31.59.193:43741)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.194@KRBTEST.COM" created.
I20260810 06:37:42.907094 4897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:39297
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
I20260810 06:37:42.929001 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40989
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:40997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:43.241050 5259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:43.241302 5259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:43.241436 5259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:43.248544 5259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:43.248644 5259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:43.248790 5259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:37:43.258086 5259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:40997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:43.259286 5259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:43.260571 5259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:43.272791 5275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:43.272884 5272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:43.272855 5273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:43.273703 5259 server_base.cc:1034] running on GCE node
I20260810 06:37:43.274266 5259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:43.275144 5259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:43.276336 5259 hybrid_clock.cc:648] HybridClock initialized: now 1786343863276290 us; error 36 us; skew 500 ppm
I20260810 06:37:43.276624 5259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:43 dist-test-slave-4tdj krb5kdc[4822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343863, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:43.279531 5259 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:37:43.280778 5259 webserver.cc:460] Webserver started at http://127.31.59.194:44469/ using document root <none> and password file <none>
I20260810 06:37:43.281371 5259 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:43.281497 5259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:43.281774 5259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:43.284494 5259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "fff31305a861419fb8953a7c51e56255"
format_stamp: "Formatted at 2026-08-10 06:37:43 on dist-test-slave-4tdj"
server_key: "47aa31e9052aa45ca0afba69b5b7c90c"
server_key_iv: "f668d62ff8d070e35aa6d9ef6391147d"
server_key_version: "encryptionkey@0"
I20260810 06:37:43.285238 5259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "fff31305a861419fb8953a7c51e56255"
format_stamp: "Formatted at 2026-08-10 06:37:43 on dist-test-slave-4tdj"
server_key: "47aa31e9052aa45ca0afba69b5b7c90c"
server_key_iv: "f668d62ff8d070e35aa6d9ef6391147d"
server_key_version: "encryptionkey@0"
I20260810 06:37:43.289920 5259 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260810 06:37:43.293079 5289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:43.294269 5259 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:37:43.294509 5259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "fff31305a861419fb8953a7c51e56255"
format_stamp: "Formatted at 2026-08-10 06:37:43 on dist-test-slave-4tdj"
server_key: "47aa31e9052aa45ca0afba69b5b7c90c"
server_key_iv: "f668d62ff8d070e35aa6d9ef6391147d"
server_key_version: "encryptionkey@0"
I20260810 06:37:43.294701 5259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:43.321870 5259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:43.325546 5259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:43.325809 5259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:43.326761 5259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:43.328011 5259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:43.328087 5259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:43.328205 5259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:43.328259 5259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:43.358217 5259 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:44915
I20260810 06:37:43.358259 5482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:44915 every 8 connection(s)
I20260810 06:37:43.359462 5259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:37:43.365409 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 5259
I20260810 06:37:43.365588 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:37:43.365978 31983 external_mini_cluster.cc:1428] Setting key 6d801bc32f008e768a8590439f9de326
Aug 10 06:37:43 dist-test-slave-4tdj krb5kdc[4822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343863, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:43.379365 5254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:43.361878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:42169 (local address 127.31.59.254:40989)
0810 06:37:43.362039 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:43.362044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:43.363169 (+ 1125us) server_negotiation.cc:407] Connection header received
0810 06:37:43.364946 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:43.364952 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:43.365042 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:43.365160 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:43.366777 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:43.367612 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:43.369304 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:43.369923 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:43.370048 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:43.373437 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:43.373442 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:43.373445 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:43.373485 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:43.376404 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:43.377090 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:43.377094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:43.377096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:43.377220 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:43.377659 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:43.377663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:43.377665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:43.377973 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:43.378114 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:43.378815 (+ 701us) server_negotiation.cc:299] Negotiation successful
0810 06:37:43.379037 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:37:43.380316 5492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:43.362281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40989 (local address 127.31.59.194:42169)
0810 06:37:43.362992 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:43.363046 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:43.364690 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:43.365389 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:43.365401 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:43.365952 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:43.366618 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:43.366638 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:43.367777 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:43.367782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:43.369131 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:43.369149 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:43.370140 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:43.370145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:43.370376 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:43.371008 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:43.371029 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:43.373220 (+ 2191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:43.376528 (+ 3308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:43.376537 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:43.376553 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:43.376959 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:43.377325 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:43.377331 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:43.377335 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:43.377543 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:43.378097 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:43.378104 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:43.378455 (+ 351us) client_negotiation.cc:769] Sending connection context
0810 06:37:43.378691 (+ 236us) client_negotiation.cc:240] Negotiation successful
0810 06:37:43.379216 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":258,"threads_started":1}
I20260810 06:37:43.381729 5486 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40989
I20260810 06:37:43.382001 5486 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:43.382741 5486 heartbeater.cc:511] Master 127.31.59.254:40989 requested a full tablet report, sending...
I20260810 06:37:43.384177 4897 ts_manager.cc:194] Registered new tserver with Master: fff31305a861419fb8953a7c51e56255 (127.31.59.194:44915)
I20260810 06:37:43.385282 4897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:42169
WARNING: no policy specified for kudu/127.31.59.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
I20260810 06:37:43.410636 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40989
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:40997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:43.736562 5496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:43.736836 5496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:43.736934 5496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:43.744151 5496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:43.744280 5496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:43.744418 5496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:37:43.753928 5496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:40997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:43.755198 5496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:43.756444 5496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:43.769428 5510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:43.769542 5509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:43.769517 5512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:43.770346 5496 server_base.cc:1034] running on GCE node
I20260810 06:37:43.770963 5496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:43.771889 5496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:43.773098 5496 hybrid_clock.cc:648] HybridClock initialized: now 1786343863773047 us; error 43 us; skew 500 ppm
I20260810 06:37:43.773411 5496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:43 dist-test-slave-4tdj krb5kdc[4822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343863, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:43.776623 5496 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:37:43.778111 5496 webserver.cc:460] Webserver started at http://127.31.59.195:43433/ using document root <none> and password file <none>
I20260810 06:37:43.778810 5496 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:43.778939 5496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:43.779253 5496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:43.782150 5496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "3dd17923dd0f4eb8a649217ce209630a"
format_stamp: "Formatted at 2026-08-10 06:37:43 on dist-test-slave-4tdj"
server_key: "d01aaca36e5ac1a14e852628e82fb8af"
server_key_iv: "fce0aa188e6fdc9b7c048134389ad5f1"
server_key_version: "encryptionkey@0"
I20260810 06:37:43.783085 5496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "3dd17923dd0f4eb8a649217ce209630a"
format_stamp: "Formatted at 2026-08-10 06:37:43 on dist-test-slave-4tdj"
server_key: "d01aaca36e5ac1a14e852628e82fb8af"
server_key_iv: "fce0aa188e6fdc9b7c048134389ad5f1"
server_key_version: "encryptionkey@0"
I20260810 06:37:43.788201 5496 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:37:43.791605 5522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:43.792975 5496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:37:43.793140 5496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "3dd17923dd0f4eb8a649217ce209630a"
format_stamp: "Formatted at 2026-08-10 06:37:43 on dist-test-slave-4tdj"
server_key: "d01aaca36e5ac1a14e852628e82fb8af"
server_key_iv: "fce0aa188e6fdc9b7c048134389ad5f1"
server_key_version: "encryptionkey@0"
I20260810 06:37:43.793331 5496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:43.823834 5496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:43.827883 5496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:43.828190 5496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:43.829212 5496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:43.830708 5496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:43.830797 5496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:43.830927 5496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:43.831000 5496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:43.864082 5496 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:38627
I20260810 06:37:43.864100 5704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:38627 every 8 connection(s)
I20260810 06:37:43.865476 5496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:37:43.868520 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 5496
I20260810 06:37:43.868718 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:37:43.869190 31983 external_mini_cluster.cc:1428] Setting key fa3086894470eb8b64af0c02c2059285
Aug 10 06:37:43 dist-test-slave-4tdj krb5kdc[4822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343863, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:43.886134 5254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:43.868099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:49319 (local address 127.31.59.254:40989)
0810 06:37:43.868315 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:43.868322 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:43.869752 (+ 1430us) server_negotiation.cc:407] Connection header received
0810 06:37:43.871577 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:43.871583 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:43.871671 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:43.871789 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:43.873224 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:43.874030 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:43.875591 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:43.876022 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:43.876137 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:43.879425 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:43.879432 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:43.879436 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:43.879477 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:43.882800 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:43.883581 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:43.883586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:43.883589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:43.883749 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:43.884200 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:43.884205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:43.884208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:43.884591 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:43.884738 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:43.885577 (+ 839us) server_negotiation.cc:299] Negotiation successful
0810 06:37:43.885809 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:37:43.887032 5708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:43.868773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40989 (local address 127.31.59.195:49319)
0810 06:37:43.869557 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:43.869620 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:43.871324 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:43.871969 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:43.871980 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:43.872516 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:43.873073 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:43.873091 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:43.874188 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:43.874192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:43.875439 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:43.875452 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:43.876218 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:43.876223 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:43.876428 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:43.877072 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:43.877094 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:43.879208 (+ 2114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:43.882928 (+ 3720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:43.882934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:43.882945 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:43.883449 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:43.883852 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:43.883855 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:43.883858 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:43.884050 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:43.884740 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:43.884747 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:43.885169 (+ 422us) client_negotiation.cc:769] Sending connection context
0810 06:37:43.885431 (+ 262us) client_negotiation.cc:240] Negotiation successful
0810 06:37:43.885922 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":316,"threads_started":1}
I20260810 06:37:43.888330 5706 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40989
I20260810 06:37:43.888615 5706 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:43.889362 5706 heartbeater.cc:511] Master 127.31.59.254:40989 requested a full tablet report, sending...
I20260810 06:37:43.890825 4897 ts_manager.cc:194] Registered new tserver with Master: 3dd17923dd0f4eb8a649217ce209630a (127.31.59.195:38627)
I20260810 06:37:43.891623 4897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:49319
I20260810 06:37:43.896664 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:37:43.910492 5250 heartbeater.cc:503] Master 127.31.59.254:40989 was elected leader, sending a full tablet report...
Aug 10 06:37:43 dist-test-slave-4tdj krb5kdc[4822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343863, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 06:37:43 dist-test-slave-4tdj krb5kdc[4822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343863, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:43.947551 5254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:43.933524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43786 (local address 127.31.59.254:40989)
0810 06:37:43.933851 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:43.933858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:43.934342 (+ 484us) server_negotiation.cc:407] Connection header received
0810 06:37:43.934602 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:43.934637 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:43.934732 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:43.934821 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:43.936328 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:43.937071 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:43.938362 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:43.938770 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:43.938861 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:43.942574 (+ 3713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:43.942578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:43.942581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:43.942615 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:43.945022 (+ 2407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:43.945641 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:43.945645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:43.945647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:43.945797 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:43.946168 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:43.946172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:43.946174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:43.946527 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:43.946768 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:43.947056 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 06:37:43.947273 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260810 06:37:43.948056 4897 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43786
I20260810 06:37:43.949951 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 5022
Aug 10 06:37:43 dist-test-slave-4tdj krb5kdc[4822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343863, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:37:43.965716 5723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:43.950940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34440 (local address 127.31.59.193:43741)
0810 06:37:43.951532 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:43.951539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:43.951563 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:37:43.951616 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:43.951623 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:43.951827 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:43.951951 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:43.953183 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:43.954202 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:43.955834 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:43.956423 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:43.956524 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:43.960490 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:43.960502 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:43.960516 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:43.960557 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:43.963150 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:43.963940 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:43.963947 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:43.963950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:43.964084 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:43.964410 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:43.964413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:43.964415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:43.964733 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:43.964944 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:43.965207 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 06:37:43.965414 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":304,"threads_started":1}
W20260810 06:37:43.967047 5243 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34440
W20260810 06:37:43.968155 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 5022: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:37:43.968320 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 5022
I20260810 06:37:44.002763 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 5259
Aug 10 06:37:44 dist-test-slave-4tdj krb5kdc[4822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343863, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.59.194@KRBTEST.COM
I20260810 06:37:44.018249 5724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:44.003588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44296 (local address 127.31.59.194:44915)
0810 06:37:44.004118 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:44.004126 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:44.004153 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:37:44.004216 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:44.004223 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:44.004432 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:44.004571 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:44.005967 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:44.007138 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:44.008408 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:44.008957 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:44.009062 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:44.013012 (+ 3950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:44.013024 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:44.013039 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:44.013077 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:44.015743 (+ 2666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:44.016339 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:44.016347 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:44.016349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:44.016475 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:44.016810 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:44.016815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:44.016817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:44.017265 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:44.017542 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:44.017725 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 06:37:44.017931 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":259,"threads_started":1}
W20260810 06:37:44.019611 5472 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44296
W20260810 06:37:44.020639 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 5259: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:37:44.020821 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 5259
I20260810 06:37:44.045732 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 5496
Aug 10 06:37:44 dist-test-slave-4tdj krb5kdc[4822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343863, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.59.195@KRBTEST.COM
I20260810 06:37:44.062703 5730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:44.046752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48074 (local address 127.31.59.195:38627)
0810 06:37:44.047416 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:44.047424 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:44.047453 (+ 29us) server_negotiation.cc:407] Connection header received
0810 06:37:44.047511 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:44.047518 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:44.047747 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:44.047895 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:44.049219 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:44.050641 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:44.051860 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:44.052406 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:44.052519 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:44.057015 (+ 4496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:44.057030 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:44.057044 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:44.057082 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:44.059924 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:44.060565 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:44.060573 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:44.060575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:44.060710 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:44.061073 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:44.061078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:44.061081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:44.061423 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:44.061733 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:44.062076 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 06:37:44.062371 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":334,"threads_started":1}
W20260810 06:37:44.064021 5699 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48074
W20260810 06:37:44.065199 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 5496: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:37:44.065361 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 5496
I20260810 06:37:44.092595 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 4844
I20260810 06:37:44.093832 4955 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:44.195926 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 4844
2026-08-10T06:37:44Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3704 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:37:44 dist-test-slave-4tdj krb5kdc[5740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:37:44 dist-test-slave-4tdj krb5kdc[5740](info): set up 2 sockets
Aug 10 06:37:44 dist-test-slave-4tdj krb5kdc[5740](info): commencing operation
krb5kdc: starting...
W20260810 06:37:45.416433 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:37:45 dist-test-slave-4tdj krb5kdc[5740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:37:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:37:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:37:45.607661 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:34939
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:42823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:34939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:45.917057 5760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:45.917320 5760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:45.917454 5760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:45.924475 5760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:37:45.924567 5760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:45.924639 5760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:45.924721 5760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:37:45.924778 5760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:37:45.934180 5760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:42823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:34939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:34939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:45.935405 5760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:45.936636 5760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:45.950465 5766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:45.950536 5768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:45.950555 5770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:45.951342 5760 server_base.cc:1034] running on GCE node
I20260810 06:37:45.952051 5760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:45.953286 5760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:45.954459 5760 hybrid_clock.cc:648] HybridClock initialized: now 1786343865954409 us; error 43 us; skew 500 ppm
I20260810 06:37:45.954778 5760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:45 dist-test-slave-4tdj krb5kdc[5740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343865, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:45.961760 5760 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:37:45.963183 5760 webserver.cc:460] Webserver started at http://127.31.59.254:34093/ using document root <none> and password file <none>
I20260810 06:37:45.963820 5760 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:45.963935 5760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:45.964195 5760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:45.967025 5760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "a9c83fb96b874d7cb9f763a16e3beee3"
format_stamp: "Formatted at 2026-08-10 06:37:45 on dist-test-slave-4tdj"
server_key: "e0cc743c3932a72f16d3d5170ec77bfa"
server_key_iv: "5d560c5ba99c903e65e8d1fe9e49de9a"
server_key_version: "encryptionkey@0"
I20260810 06:37:45.967746 5760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "a9c83fb96b874d7cb9f763a16e3beee3"
format_stamp: "Formatted at 2026-08-10 06:37:45 on dist-test-slave-4tdj"
server_key: "e0cc743c3932a72f16d3d5170ec77bfa"
server_key_iv: "5d560c5ba99c903e65e8d1fe9e49de9a"
server_key_version: "encryptionkey@0"
I20260810 06:37:45.972460 5760 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:37:45.975710 5777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:45.976900 5760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:37:45.977056 5760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "a9c83fb96b874d7cb9f763a16e3beee3"
format_stamp: "Formatted at 2026-08-10 06:37:45 on dist-test-slave-4tdj"
server_key: "e0cc743c3932a72f16d3d5170ec77bfa"
server_key_iv: "5d560c5ba99c903e65e8d1fe9e49de9a"
server_key_version: "encryptionkey@0"
I20260810 06:37:45.977228 5760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:45.991305 5760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:45.995049 5760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:45.995283 5760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:46.012104 5760 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:34939
I20260810 06:37:46.012132 5878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:34939 every 8 connection(s)
I20260810 06:37:46.013376 5760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:37:46.017026 5879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:46.023711 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 5760
I20260810 06:37:46.023890 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:37:46.024488 31983 external_mini_cluster.cc:1428] Setting key cae65e1613188d053cf9ff3d24ed51d0
I20260810 06:37:46.026970 5879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3: Bootstrap starting.
I20260810 06:37:46.030124 5879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:46.031405 5879 log.cc:824] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:46.034409 5879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3: No bootstrap required, opened a new log
Aug 10 06:37:46 dist-test-slave-4tdj krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:46.043008 5879 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c83fb96b874d7cb9f763a16e3beee3" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34939 } }
I20260810 06:37:46.043403 5879 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:46.043514 5879 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c83fb96b874d7cb9f763a16e3beee3, State: Initialized, Role: FOLLOWER
I20260810 06:37:46.044070 5879 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c83fb96b874d7cb9f763a16e3beee3" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34939 } }
I20260810 06:37:46.044349 5879 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:37:46.044517 5879 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:37:46.044687 5879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:46.047015 5883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:46.026337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60772 (local address 127.31.59.254:34939)
0810 06:37:46.027091 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:46.027106 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:46.027154 (+ 48us) server_negotiation.cc:407] Connection header received
0810 06:37:46.028795 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:46.028815 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:46.029161 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:46.029454 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:46.031363 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:46.033041 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:46.034785 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:46.035431 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:46.035573 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:46.040064 (+ 4491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:46.040071 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:46.040088 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:46.040132 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:46.043056 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:46.043783 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:46.043792 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:46.043799 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:46.044018 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:46.044404 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:46.044409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:46.044413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:46.044980 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:46.045266 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:46.045657 (+ 391us) server_negotiation.cc:299] Negotiation successful
0810 06:37:46.045980 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":352,"threads_started":1}
I20260810 06:37:46.047254 5879 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c83fb96b874d7cb9f763a16e3beee3" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34939 } }
I20260810 06:37:46.047735 5879 leader_election.cc:302] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9c83fb96b874d7cb9f763a16e3beee3; no voters:
I20260810 06:37:46.048316 5879 leader_election.cc:288] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:37:46.048434 5887 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:46.049553 5887 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3 [term 1 LEADER]: Becoming Leader. State: Replica: a9c83fb96b874d7cb9f763a16e3beee3, State: Running, Role: LEADER
I20260810 06:37:46.050129 5887 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c83fb96b874d7cb9f763a16e3beee3" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34939 } }
I20260810 06:37:46.050552 5879 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:37:46.052856 5892 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9c83fb96b874d7cb9f763a16e3beee3. Latest consensus state: current_term: 1 leader_uuid: "a9c83fb96b874d7cb9f763a16e3beee3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c83fb96b874d7cb9f763a16e3beee3" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34939 } } }
I20260810 06:37:46.052919 5888 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9c83fb96b874d7cb9f763a16e3beee3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c83fb96b874d7cb9f763a16e3beee3" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34939 } } }
I20260810 06:37:46.053248 5892 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:46.053257 5888 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:46.062151 5897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:37:46.066468 5897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:37:46.075155 5897 catalog_manager.cc:1347] Generated new cluster ID: 9c29564b269f47e4bf2906eb97098f79
I20260810 06:37:46.075287 5897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:37:46.087880 5897 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:37:46.089082 5897 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:37:46.095323 5897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9c83fb96b874d7cb9f763a16e3beee3: Generated new TSK 0
I20260810 06:37:46.096027 5897 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
I20260810 06:37:46.145402 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:34939
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:42823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:46.463778 5924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:46.464035 5924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:46.464138 5924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:46.471151 5924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:46.471256 5924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:46.471408 5924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:37:46.480863 5924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:42823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:34939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:46.482160 5924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:46.483495 5924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:46.495921 5933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:46.495931 5935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:46.496098 5937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:46.496767 5924 server_base.cc:1034] running on GCE node
I20260810 06:37:46.497360 5924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:46.498261 5924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:46.499486 5924 hybrid_clock.cc:648] HybridClock initialized: now 1786343866499432 us; error 47 us; skew 500 ppm
I20260810 06:37:46.499797 5924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:46 dist-test-slave-4tdj krb5kdc[5740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343866, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:46.502817 5924 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:37:46.504164 5924 webserver.cc:460] Webserver started at http://127.31.59.193:38091/ using document root <none> and password file <none>
I20260810 06:37:46.504823 5924 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:46.504941 5924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:46.505247 5924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:46.508111 5924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "bbc93c98e24742a1a3788d26f75c3cee"
format_stamp: "Formatted at 2026-08-10 06:37:46 on dist-test-slave-4tdj"
server_key: "846de9f684ef96763bbb626f90b45c18"
server_key_iv: "605e98e2bdf905e08bed8bb13c32c164"
server_key_version: "encryptionkey@0"
I20260810 06:37:46.508873 5924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "bbc93c98e24742a1a3788d26f75c3cee"
format_stamp: "Formatted at 2026-08-10 06:37:46 on dist-test-slave-4tdj"
server_key: "846de9f684ef96763bbb626f90b45c18"
server_key_iv: "605e98e2bdf905e08bed8bb13c32c164"
server_key_version: "encryptionkey@0"
I20260810 06:37:46.513753 5924 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:37:46.517191 5945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:46.518465 5924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:37:46.518638 5924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "bbc93c98e24742a1a3788d26f75c3cee"
format_stamp: "Formatted at 2026-08-10 06:37:46 on dist-test-slave-4tdj"
server_key: "846de9f684ef96763bbb626f90b45c18"
server_key_iv: "605e98e2bdf905e08bed8bb13c32c164"
server_key_version: "encryptionkey@0"
I20260810 06:37:46.518826 5924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:46.531019 5924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:46.534824 5924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:46.535081 5924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:46.535956 5924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:46.537218 5924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:46.537294 5924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:46.537402 5924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:46.537454 5924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:46.567139 5924 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:42721
I20260810 06:37:46.567232 6124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:42721 every 8 connection(s)
I20260810 06:37:46.568367 5924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:37:46.573064 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 5924
I20260810 06:37:46.573244 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:37:46.573861 31983 external_mini_cluster.cc:1428] Setting key ae47c3dcaec5bc5c11914845ba9e7632
Aug 10 06:37:46 dist-test-slave-4tdj krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343866, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:46.588078 6129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:46.570791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:57729 (local address 127.31.59.254:34939)
0810 06:37:46.571188 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:46.571192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:46.572085 (+ 893us) server_negotiation.cc:407] Connection header received
0810 06:37:46.573912 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:46.573917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:46.573996 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:46.574094 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:46.575805 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:46.576496 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:46.578000 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:46.578441 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:46.578560 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:46.582130 (+ 3570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:46.582135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:46.582138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:46.582171 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:46.585146 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:46.585841 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:46.585847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:46.585850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:46.586004 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:46.586413 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:46.586417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:46.586420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:46.586765 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:46.586907 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:46.587537 (+ 630us) server_negotiation.cc:299] Negotiation successful
0810 06:37:46.587746 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":211,"threads_started":1}
I20260810 06:37:46.588807 6131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:46.571254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:34939 (local address 127.31.59.193:57729)
0810 06:37:46.571921 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:46.571965 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:46.573642 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:46.574361 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:46.574370 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:46.574976 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:46.575647 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:46.575668 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:46.576659 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:46.576663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:46.577858 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:46.577871 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:46.578669 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:46.578677 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:46.578878 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:46.579490 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:46.579514 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:46.581928 (+ 2414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:46.585282 (+ 3354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:46.585288 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:46.585299 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:46.585719 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:46.586102 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:46.586105 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:46.586108 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:46.586279 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:46.586872 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:46.586877 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:46.587161 (+ 284us) client_negotiation.cc:769] Sending connection context
0810 06:37:46.587393 (+ 232us) client_negotiation.cc:240] Negotiation successful
0810 06:37:46.587762 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":253,"threads_started":1}
I20260810 06:37:46.590178 6126 heartbeater.cc:347] Connected to a master server at 127.31.59.254:34939
I20260810 06:37:46.590536 6126 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:46.591408 6126 heartbeater.cc:511] Master 127.31.59.254:34939 requested a full tablet report, sending...
I20260810 06:37:46.593607 5812 ts_manager.cc:194] Registered new tserver with Master: bbc93c98e24742a1a3788d26f75c3cee (127.31.59.193:42721)
WARNING: no policy specified for kudu/127.31.59.194@KRBTEST.COM; defaulting to no policy
I20260810 06:37:46.595294 5812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:57729
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
I20260810 06:37:46.619604 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:34939
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:42823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:46.931531 6138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:46.931811 6138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:46.931946 6138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:46.939250 6138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:46.939361 6138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:46.939517 6138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:37:46.949079 6138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:42823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:34939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:46.950314 6138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:46.951678 6138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:46.964306 6146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:46.964272 6152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:46.964378 6147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:46.965044 6138 server_base.cc:1034] running on GCE node
I20260810 06:37:46.965618 6138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:46.966463 6138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:46.967649 6138 hybrid_clock.cc:648] HybridClock initialized: now 1786343866967600 us; error 40 us; skew 500 ppm
I20260810 06:37:46.967921 6138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:46 dist-test-slave-4tdj krb5kdc[5740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343866, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:46.970820 6138 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:37:46.972177 6138 webserver.cc:460] Webserver started at http://127.31.59.194:34087/ using document root <none> and password file <none>
I20260810 06:37:46.972817 6138 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:46.972934 6138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:46.973227 6138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:46.975983 6138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "8bcdc43b82434ddcaf963c9f40a22e0d"
format_stamp: "Formatted at 2026-08-10 06:37:46 on dist-test-slave-4tdj"
server_key: "fa3f47ba5a4dd352c1a6b51d9ef9bc6b"
server_key_iv: "728eeaf43bdfe09bfcfd4c9cfc0e7b7d"
server_key_version: "encryptionkey@0"
I20260810 06:37:46.976770 6138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "8bcdc43b82434ddcaf963c9f40a22e0d"
format_stamp: "Formatted at 2026-08-10 06:37:46 on dist-test-slave-4tdj"
server_key: "fa3f47ba5a4dd352c1a6b51d9ef9bc6b"
server_key_iv: "728eeaf43bdfe09bfcfd4c9cfc0e7b7d"
server_key_version: "encryptionkey@0"
I20260810 06:37:46.981899 6138 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:37:46.985304 6162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:46.986639 6138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:37:46.986794 6138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "8bcdc43b82434ddcaf963c9f40a22e0d"
format_stamp: "Formatted at 2026-08-10 06:37:46 on dist-test-slave-4tdj"
server_key: "fa3f47ba5a4dd352c1a6b51d9ef9bc6b"
server_key_iv: "728eeaf43bdfe09bfcfd4c9cfc0e7b7d"
server_key_version: "encryptionkey@0"
I20260810 06:37:46.986975 6138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:47.003752 6138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:47.007361 6138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:47.007613 6138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:47.008453 6138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:47.009712 6138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:47.009789 6138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:47.009905 6138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:47.009958 6138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:47.045657 6138 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:40015
I20260810 06:37:47.045711 6346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:40015 every 8 connection(s)
I20260810 06:37:47.046902 6138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:37:47.055083 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 6138
I20260810 06:37:47.055238 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:37:47.055653 31983 external_mini_cluster.cc:1428] Setting key d0156d907067f978eb8c9f37b4d39641
Aug 10 06:37:47 dist-test-slave-4tdj krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343866, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:47.065861 6129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:47.049268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:56053 (local address 127.31.59.254:34939)
0810 06:37:47.049452 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:47.049457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:47.050570 (+ 1113us) server_negotiation.cc:407] Connection header received
0810 06:37:47.052359 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:47.052363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:47.052430 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:47.052522 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:47.053894 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:47.054656 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:47.056220 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:47.056731 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:47.056827 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:47.060316 (+ 3489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:47.060323 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:47.060328 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:47.060364 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:47.063082 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:47.063739 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:47.063745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:47.063749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:47.063879 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:47.064276 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:47.064281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:47.064283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:47.064531 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:47.064672 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:47.065314 (+ 642us) server_negotiation.cc:299] Negotiation successful
0810 06:37:47.065542 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 06:37:47.066787 6350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:47.049701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:34939 (local address 127.31.59.194:56053)
0810 06:37:47.050424 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:47.050465 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:47.052134 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:47.052717 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:47.052725 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:47.053221 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:47.053746 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:47.053763 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:47.054784 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:47.054788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:47.056070 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:47.056083 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:47.056953 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:47.056960 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:47.057237 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:47.057928 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:47.057954 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:47.060113 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:47.063212 (+ 3099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:47.063221 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:47.063237 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:47.063626 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:47.063982 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:47.063996 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:47.064000 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:47.064171 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:47.064652 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:47.064659 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:47.064956 (+ 297us) client_negotiation.cc:769] Sending connection context
0810 06:37:47.065231 (+ 275us) client_negotiation.cc:240] Negotiation successful
0810 06:37:47.065698 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":259,"threads_started":1}
I20260810 06:37:47.068117 6347 heartbeater.cc:347] Connected to a master server at 127.31.59.254:34939
I20260810 06:37:47.068380 6347 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:47.069060 6347 heartbeater.cc:511] Master 127.31.59.254:34939 requested a full tablet report, sending...
I20260810 06:37:47.070394 5812 ts_manager.cc:194] Registered new tserver with Master: 8bcdc43b82434ddcaf963c9f40a22e0d (127.31.59.194:40015)
I20260810 06:37:47.071156 5812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:56053
WARNING: no policy specified for kudu/127.31.59.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
I20260810 06:37:47.099584 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:34939
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:42823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:47.409982 6355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:47.410238 6355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:47.410364 6355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:47.417326 6355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:47.417421 6355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:47.417546 6355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:37:47.426964 6355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:42823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:34939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:47.428120 6355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:47.429358 6355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:47.441582 6365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:47.441660 6361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:47.441680 6369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:47.442636 6355 server_base.cc:1034] running on GCE node
I20260810 06:37:47.443192 6355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:47.444075 6355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:47.445302 6355 hybrid_clock.cc:648] HybridClock initialized: now 1786343867445252 us; error 42 us; skew 500 ppm
I20260810 06:37:47.445655 6355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:47 dist-test-slave-4tdj krb5kdc[5740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343867, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:47.448730 6355 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:37:47.450146 6355 webserver.cc:460] Webserver started at http://127.31.59.195:33139/ using document root <none> and password file <none>
I20260810 06:37:47.450821 6355 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:47.450940 6355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:47.451268 6355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:47.454027 6355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "8af31430ce63478b919bd70594c0692f"
format_stamp: "Formatted at 2026-08-10 06:37:47 on dist-test-slave-4tdj"
server_key: "929e8ab4e2ee00916a8c76251db630df"
server_key_iv: "da7ee400f32f64dd31641d343eedc62b"
server_key_version: "encryptionkey@0"
I20260810 06:37:47.454902 6355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "8af31430ce63478b919bd70594c0692f"
format_stamp: "Formatted at 2026-08-10 06:37:47 on dist-test-slave-4tdj"
server_key: "929e8ab4e2ee00916a8c76251db630df"
server_key_iv: "da7ee400f32f64dd31641d343eedc62b"
server_key_version: "encryptionkey@0"
I20260810 06:37:47.460726 6355 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:37:47.464071 6385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:47.465345 6355 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:37:47.465484 6355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "8af31430ce63478b919bd70594c0692f"
format_stamp: "Formatted at 2026-08-10 06:37:47 on dist-test-slave-4tdj"
server_key: "929e8ab4e2ee00916a8c76251db630df"
server_key_iv: "da7ee400f32f64dd31641d343eedc62b"
server_key_version: "encryptionkey@0"
I20260810 06:37:47.465670 6355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:47.476409 6355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:47.480129 6355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:47.480384 6355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:47.481287 6355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:47.482609 6355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:47.482681 6355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:47.482769 6355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:47.482810 6355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:47.513352 6355 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:39859
I20260810 06:37:47.513388 6561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:39859 every 8 connection(s)
I20260810 06:37:47.514643 6355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:37:47.516189 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 6355
I20260810 06:37:47.516361 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:37:47.516891 31983 external_mini_cluster.cc:1428] Setting key b8b4a09ec8c42abb40a65c0f379c1af5
Aug 10 06:37:47 dist-test-slave-4tdj krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343867, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:47.533389 6129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:47.517398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:43153 (local address 127.31.59.254:34939)
0810 06:37:47.517618 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:47.517624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:47.518958 (+ 1334us) server_negotiation.cc:407] Connection header received
0810 06:37:47.520771 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:47.520775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:47.520841 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:47.520912 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:47.522281 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:47.523103 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:47.524671 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:47.525066 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:47.525155 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:47.528291 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:47.528296 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:47.528300 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:47.528338 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:47.530613 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:47.531406 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:47.531411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:47.531414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:47.531580 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:47.532050 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:47.532053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:47.532055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:47.532318 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:47.532426 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:47.532982 (+ 556us) server_negotiation.cc:299] Negotiation successful
0810 06:37:47.533140 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:37:47.534464 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:47.517998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:34939 (local address 127.31.59.195:43153)
0810 06:37:47.518818 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:47.518863 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:47.520514 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:47.521083 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:47.521090 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:47.521579 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:47.522126 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:47.522143 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:47.523243 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:47.523247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:47.524546 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:47.524564 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:47.525235 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:47.525240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:47.525427 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:47.526001 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:47.526021 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:47.528095 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:47.530728 (+ 2633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:47.530736 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:47.530750 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:47.531290 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:47.531698 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:47.531703 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:47.531707 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:47.531941 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:47.532414 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:47.532420 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:47.532699 (+ 279us) client_negotiation.cc:769] Sending connection context
0810 06:37:47.532916 (+ 217us) client_negotiation.cc:240] Negotiation successful
0810 06:37:47.533267 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":329,"threads_started":1}
I20260810 06:37:47.535640 6562 heartbeater.cc:347] Connected to a master server at 127.31.59.254:34939
I20260810 06:37:47.535892 6562 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:47.536511 6562 heartbeater.cc:511] Master 127.31.59.254:34939 requested a full tablet report, sending...
I20260810 06:37:47.537766 5812 ts_manager.cc:194] Registered new tserver with Master: 8af31430ce63478b919bd70594c0692f (127.31.59.195:39859)
I20260810 06:37:47.538578 5812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:43153
I20260810 06:37:47.543819 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 06:37:47 dist-test-slave-4tdj krb5kdc[5740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:37:47 dist-test-slave-4tdj krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:37:47.591427 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:47.577329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44548 (local address 127.31.59.193:42721)
0810 06:37:47.577906 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:47.577915 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:47.577943 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:37:47.578013 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:47.578019 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:47.578209 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:47.578379 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:47.579751 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:47.580740 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:47.581971 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:47.582560 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:47.582688 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:47.586206 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:47.586215 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:47.586228 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:47.586257 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:47.588705 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:47.589318 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:47.589325 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:47.589327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:47.589464 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:47.589836 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:47.589839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:47.589841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:47.590193 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:47.590512 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:47.590802 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 06:37:47.591063 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":274,"threads_started":1}
I20260810 06:37:47.597991 6126 heartbeater.cc:503] Master 127.31.59.254:34939 was elected leader, sending a full tablet report...
Aug 10 06:37:47 dist-test-slave-4tdj krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:47.607615 6129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:47.595280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60774 (local address 127.31.59.254:34939)
0810 06:37:47.595479 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:47.595485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:47.595639 (+ 154us) server_negotiation.cc:407] Connection header received
0810 06:37:47.595791 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:47.595796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:47.595858 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:47.595939 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:47.597153 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:47.597850 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:47.599263 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:47.599730 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:47.599830 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:47.603212 (+ 3382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:47.603217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:47.603220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:47.603251 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:47.605512 (+ 2261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:47.606032 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:47.606036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:47.606038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:47.606169 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:47.606480 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:47.606483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:47.606485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:47.606762 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:47.606938 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:47.607176 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 06:37:47.607327 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 06:37:47.608251 5812 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60774
I20260810 06:37:47.609217 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 5924
I20260810 06:37:47.620095 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:47.609992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44550 (local address 127.31.59.193:42721)
0810 06:37:47.610132 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:47.610136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:47.610368 (+ 232us) server_negotiation.cc:407] Connection header received
0810 06:37:47.610496 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:47.610500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:47.610554 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:47.610627 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:47.611687 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:47.612411 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:47.613577 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:47.613983 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:47.614081 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:47.616000 (+ 1919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:47.616004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:47.616008 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:47.616037 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:47.618203 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:47.618739 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:47.618742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:47.618744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:47.618855 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:47.619125 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:47.619128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:47.619130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:47.619340 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:47.619508 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:47.619723 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 06:37:47.619872 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:37:47.620652 6119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:47.706993 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 5924
I20260810 06:37:47.730163 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 6138
Aug 10 06:37:47 dist-test-slave-4tdj krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.194@KRBTEST.COM
I20260810 06:37:47.746788 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:47.731110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47790 (local address 127.31.59.194:40015)
0810 06:37:47.731660 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:47.731667 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:47.731693 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:37:47.731761 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:47.731767 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:47.731976 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:47.732104 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:47.733426 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:47.734584 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:47.735848 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:47.736410 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:47.736510 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:47.740859 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:47.740871 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:47.740885 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:47.740921 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:47.744135 (+ 3214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:47.744736 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:47.744743 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:47.744745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:47.744887 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:47.745202 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:47.745206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:47.745208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:47.745606 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:47.745873 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:47.746230 (+ 357us) server_negotiation.cc:299] Negotiation successful
0810 06:37:47.746487 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":247,"threads_started":1}
I20260810 06:37:47.748071 6335 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:47.828941 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 6138
I20260810 06:37:47.851867 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 6355
Aug 10 06:37:47 dist-test-slave-4tdj krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.195@KRBTEST.COM
I20260810 06:37:47.867463 6581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:47.852890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60676 (local address 127.31.59.195:39859)
0810 06:37:47.853503 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:47.853537 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:47.853567 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:37:47.853627 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:47.853634 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:47.853843 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:47.853965 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:47.855250 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:47.856286 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:47.857516 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:47.858068 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:47.858171 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:47.862267 (+ 4096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:47.862278 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:47.862317 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:47.862355 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:47.864974 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:47.865585 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:47.865592 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:47.865594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:47.865709 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:47.865963 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:47.865966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:47.865968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:47.866285 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:47.866696 (+ 411us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:47.866911 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 06:37:47.867115 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":298,"threads_started":1}
I20260810 06:37:47.868817 6555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:47.949323 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 6355
I20260810 06:37:47.969094 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 5760
I20260810 06:37:47.970247 5869 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:48.071441 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 5760
2026-08-10T06:37:48Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3872 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:37:48 dist-test-slave-4tdj krb5kdc[6592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:37:48 dist-test-slave-4tdj krb5kdc[6592](info): set up 2 sockets
Aug 10 06:37:48 dist-test-slave-4tdj krb5kdc[6592](info): commencing operation
krb5kdc: starting...
W20260810 06:37:49.289767 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:37:49 dist-test-slave-4tdj krb5kdc[6592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:37:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:37:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:37:49.483744 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:36905
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:35573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:36905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:49.791935 6612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:49.792199 6612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:49.792294 6612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:49.799160 6612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:37:49.799252 6612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:49.799302 6612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:49.799355 6612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:37:49.799400 6612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:37:49.808609 6612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:35573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:36905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:36905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:49.809691 6612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:49.810895 6612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:49.821617 6621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:49.821655 6619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:49.821691 6618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:49.822387 6612 server_base.cc:1034] running on GCE node
I20260810 06:37:49.823119 6612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:49.824353 6612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:49.825558 6612 hybrid_clock.cc:648] HybridClock initialized: now 1786343869825504 us; error 47 us; skew 500 ppm
I20260810 06:37:49.825874 6612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:49 dist-test-slave-4tdj krb5kdc[6592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343869, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:49.828889 6612 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:37:49.829942 6612 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:49.830052 6612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:49.830376 6612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:49.833146 6612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "0021da411821463c95d67e7203641266"
format_stamp: "Formatted at 2026-08-10 06:37:49 on dist-test-slave-4tdj"
server_key: "2668bf3c58ef62b938230477fe9f4202"
server_key_iv: "4e8cb460672decc7d010daa4db7c0558"
server_key_version: "encryptionkey@0"
I20260810 06:37:49.833889 6612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "0021da411821463c95d67e7203641266"
format_stamp: "Formatted at 2026-08-10 06:37:49 on dist-test-slave-4tdj"
server_key: "2668bf3c58ef62b938230477fe9f4202"
server_key_iv: "4e8cb460672decc7d010daa4db7c0558"
server_key_version: "encryptionkey@0"
I20260810 06:37:49.838562 6612 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.003s
I20260810 06:37:49.841739 6629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:49.842972 6612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:37:49.843148 6612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "0021da411821463c95d67e7203641266"
format_stamp: "Formatted at 2026-08-10 06:37:49 on dist-test-slave-4tdj"
server_key: "2668bf3c58ef62b938230477fe9f4202"
server_key_iv: "4e8cb460672decc7d010daa4db7c0558"
server_key_version: "encryptionkey@0"
I20260810 06:37:49.843343 6612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:49.857937 6612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:49.861505 6612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:49.861711 6612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:49.878051 6612 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:36905
I20260810 06:37:49.878062 6708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:36905 every 8 connection(s)
I20260810 06:37:49.878887 6612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:37:49.882570 6709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:49.889163 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 6612
I20260810 06:37:49.889369 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:37:49.890141 31983 external_mini_cluster.cc:1428] Setting key 0c42951672c5489312092e5dd4b56828
I20260810 06:37:49.893229 6709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266: Bootstrap starting.
I20260810 06:37:49.896672 6709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:49.897823 6709 log.cc:824] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:49.900697 6709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266: No bootstrap required, opened a new log
Aug 10 06:37:49 dist-test-slave-4tdj krb5kdc[6592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:49.909467 6709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0021da411821463c95d67e7203641266" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36905 } }
I20260810 06:37:49.909912 6709 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:49.910017 6709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0021da411821463c95d67e7203641266, State: Initialized, Role: FOLLOWER
I20260810 06:37:49.910636 6709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0021da411821463c95d67e7203641266" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36905 } }
I20260810 06:37:49.910857 6709 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:37:49.910992 6709 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:37:49.911180 6709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:49.912809 6715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:49.892146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35180 (local address 127.31.59.254:36905)
0810 06:37:49.892875 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:49.892889 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:49.892929 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:37:49.894694 (+ 1765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:49.894716 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:49.895066 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:49.895359 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:49.897215 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:49.898649 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:49.899972 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:49.900949 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:49.901130 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:49.904822 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:49.904836 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:49.904855 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:49.904906 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:49.908790 (+ 3884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:49.909410 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:49.909419 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:49.909426 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:49.909661 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:49.910058 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:49.910062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:49.910064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:49.910682 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:49.910950 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:49.911414 (+ 464us) server_negotiation.cc:299] Negotiation successful
0810 06:37:49.911823 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":329,"threads_started":1}
I20260810 06:37:49.913563 6709 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0021da411821463c95d67e7203641266" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36905 } }
I20260810 06:37:49.914083 6709 leader_election.cc:302] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0021da411821463c95d67e7203641266; no voters:
I20260810 06:37:49.914768 6709 leader_election.cc:288] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:37:49.914860 6718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:49.916314 6718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266 [term 1 LEADER]: Becoming Leader. State: Replica: 0021da411821463c95d67e7203641266, State: Running, Role: LEADER
I20260810 06:37:49.916714 6709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:37:49.916785 6718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0021da411821463c95d67e7203641266" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36905 } }
I20260810 06:37:49.919459 6720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0021da411821463c95d67e7203641266. Latest consensus state: current_term: 1 leader_uuid: "0021da411821463c95d67e7203641266" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0021da411821463c95d67e7203641266" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36905 } } }
I20260810 06:37:49.919576 6719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0021da411821463c95d67e7203641266" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0021da411821463c95d67e7203641266" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36905 } } }
I20260810 06:37:49.919848 6719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:49.919852 6720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:49.922237 6729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:37:49.927743 6729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:37:49.936506 6729 catalog_manager.cc:1347] Generated new cluster ID: a269830769164a98a80b1af1959eee97
I20260810 06:37:49.936627 6729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:37:49.962850 6729 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:37:49.964205 6729 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:37:49.972538 6729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266: Generated new TSK 0
I20260810 06:37:49.973307 6729 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
I20260810 06:37:50.029891 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36905
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:35573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:50.344156 6755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:50.344435 6755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:50.344604 6755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:50.351826 6755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:50.351951 6755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:50.352108 6755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:37:50.361536 6755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:35573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:50.362852 6755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:50.364154 6755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:50.376282 6768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:50.376230 6766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:50.376309 6763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:50.377046 6755 server_base.cc:1034] running on GCE node
I20260810 06:37:50.377604 6755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:50.378482 6755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:50.379678 6755 hybrid_clock.cc:648] HybridClock initialized: now 1786343870379632 us; error 46 us; skew 500 ppm
I20260810 06:37:50.379962 6755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:50 dist-test-slave-4tdj krb5kdc[6592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343870, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:50.382827 6755 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:37:50.383882 6755 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:50.383983 6755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:50.384320 6755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:50.387111 6755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "e901ca39bfc148169a17a83d8ab93097"
format_stamp: "Formatted at 2026-08-10 06:37:50 on dist-test-slave-4tdj"
server_key: "0d126ed4271577058971b6fb0b64f8a9"
server_key_iv: "f20bcf9601ce9eef68d7ebaeedaac4ad"
server_key_version: "encryptionkey@0"
I20260810 06:37:50.387876 6755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "e901ca39bfc148169a17a83d8ab93097"
format_stamp: "Formatted at 2026-08-10 06:37:50 on dist-test-slave-4tdj"
server_key: "0d126ed4271577058971b6fb0b64f8a9"
server_key_iv: "f20bcf9601ce9eef68d7ebaeedaac4ad"
server_key_version: "encryptionkey@0"
I20260810 06:37:50.392644 6755 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:37:50.395876 6776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:50.397063 6755 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:37:50.397224 6755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "e901ca39bfc148169a17a83d8ab93097"
format_stamp: "Formatted at 2026-08-10 06:37:50 on dist-test-slave-4tdj"
server_key: "0d126ed4271577058971b6fb0b64f8a9"
server_key_iv: "f20bcf9601ce9eef68d7ebaeedaac4ad"
server_key_version: "encryptionkey@0"
I20260810 06:37:50.397416 6755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:50.415758 6755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:50.419504 6755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:50.419736 6755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:50.420605 6755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:50.421919 6755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:50.422021 6755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:50.422123 6755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:50.422199 6755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:50.452167 6755 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:46435
I20260810 06:37:50.452229 6969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:46435 every 8 connection(s)
I20260810 06:37:50.452991 6755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:37:50.459023 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 6755
I20260810 06:37:50.459204 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:37:50.459798 31983 external_mini_cluster.cc:1428] Setting key 273844fe0d3f5d2fa35b9cd1214ed283
Aug 10 06:37:50 dist-test-slave-4tdj krb5kdc[6592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343870, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:50.473424 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:50.455292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:58743 (local address 127.31.59.254:36905)
0810 06:37:50.455708 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:50.455712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:50.456636 (+ 924us) server_negotiation.cc:407] Connection header received
0810 06:37:50.458414 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:50.458418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:50.458486 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:50.458585 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:50.460045 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:50.461001 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:50.462755 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:50.463344 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:50.463473 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:50.467600 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:50.467606 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:50.467610 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:50.467649 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:50.470273 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:50.471127 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:50.471131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:50.471133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:50.471255 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:50.471718 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:50.471723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:50.471726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:50.472005 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:50.472118 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:50.472990 (+ 872us) server_negotiation.cc:299] Negotiation successful
0810 06:37:50.473173 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":216,"threads_started":1}
I20260810 06:37:50.474823 6978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:50.455803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36905 (local address 127.31.59.193:58743)
0810 06:37:50.456459 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:50.456504 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:50.458145 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:50.458775 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:50.458782 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:50.459330 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:50.459866 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:50.459892 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:50.461154 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:50.461159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:50.462612 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:50.462624 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:50.463574 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:50.463580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:50.463770 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:50.464433 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:50.464468 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:50.467352 (+ 2884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:50.470428 (+ 3076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:50.470437 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:50.470453 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:50.471001 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:50.471344 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:50.471349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:50.471353 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:50.471589 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:50.472132 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:50.472139 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:50.472582 (+ 443us) client_negotiation.cc:769] Sending connection context
0810 06:37:50.472888 (+ 306us) client_negotiation.cc:240] Negotiation successful
0810 06:37:50.473397 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":266,"threads_started":1}
I20260810 06:37:50.476468 6971 heartbeater.cc:347] Connected to a master server at 127.31.59.254:36905
I20260810 06:37:50.476800 6971 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:50.477602 6971 heartbeater.cc:511] Master 127.31.59.254:36905 requested a full tablet report, sending...
I20260810 06:37:50.480013 6658 ts_manager.cc:194] Registered new tserver with Master: e901ca39bfc148169a17a83d8ab93097 (127.31.59.193:46435)
I20260810 06:37:50.482065 6658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:58743
WARNING: no policy specified for kudu/127.31.59.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
I20260810 06:37:50.508670 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36905
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:35573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:50.820933 6982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:50.821195 6982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:50.821336 6982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:50.828445 6982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:50.828567 6982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:50.828722 6982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:37:50.838145 6982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:35573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:50.839465 6982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:50.840781 6982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:50.852854 6996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:50.852893 6999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:50.852816 7001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:50.859119 6982 server_base.cc:1034] running on GCE node
I20260810 06:37:50.859694 6982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:50.860554 6982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:50.861779 6982 hybrid_clock.cc:648] HybridClock initialized: now 1786343870861733 us; error 32 us; skew 500 ppm
I20260810 06:37:50.862082 6982 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:50 dist-test-slave-4tdj krb5kdc[6592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343870, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:50.868361 6982 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:37:50.869449 6982 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:50.869556 6982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:50.869866 6982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:50.872732 6982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "ae286d56dd4d4e24afb779ea610d4303"
format_stamp: "Formatted at 2026-08-10 06:37:50 on dist-test-slave-4tdj"
server_key: "aaf5dd8104644d1b124689bd020b7793"
server_key_iv: "2e5724b96839aa144a741463bb97268f"
server_key_version: "encryptionkey@0"
I20260810 06:37:50.873510 6982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "ae286d56dd4d4e24afb779ea610d4303"
format_stamp: "Formatted at 2026-08-10 06:37:50 on dist-test-slave-4tdj"
server_key: "aaf5dd8104644d1b124689bd020b7793"
server_key_iv: "2e5724b96839aa144a741463bb97268f"
server_key_version: "encryptionkey@0"
I20260810 06:37:50.878604 6982 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 06:37:50.882793 7010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:50.884433 6982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:37:50.884636 6982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "ae286d56dd4d4e24afb779ea610d4303"
format_stamp: "Formatted at 2026-08-10 06:37:50 on dist-test-slave-4tdj"
server_key: "aaf5dd8104644d1b124689bd020b7793"
server_key_iv: "2e5724b96839aa144a741463bb97268f"
server_key_version: "encryptionkey@0"
I20260810 06:37:50.884876 6982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:50.904210 6982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:50.908190 6982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:50.908414 6982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:50.909266 6982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:50.910591 6982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:50.910676 6982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:50.910753 6982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:50.910805 6982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:50.940305 6982 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:44857
I20260810 06:37:50.940362 7196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:44857 every 8 connection(s)
I20260810 06:37:50.941118 6982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:37:50.946185 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 6982
I20260810 06:37:50.946360 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:37:50.946811 31983 external_mini_cluster.cc:1428] Setting key 80dff7ab2e4e6731386ca39728215db9
Aug 10 06:37:50 dist-test-slave-4tdj krb5kdc[6592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343870, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:50.961627 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:50.943641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:37833 (local address 127.31.59.254:36905)
0810 06:37:50.943855 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:50.943861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:50.944977 (+ 1116us) server_negotiation.cc:407] Connection header received
0810 06:37:50.946904 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:50.946910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:50.946999 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:50.947115 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:50.948959 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:50.950001 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:50.951562 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:50.952088 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:50.952212 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:50.955728 (+ 3516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:50.955733 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:50.955737 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:50.955768 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:50.958385 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:50.959232 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:50.959236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:50.959239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:50.959371 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:50.959890 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:50.959894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:50.959896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:50.960191 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:50.960310 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:50.961197 (+ 887us) server_negotiation.cc:299] Negotiation successful
0810 06:37:50.961374 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:37:50.963043 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:50.944096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36905 (local address 127.31.59.194:37833)
0810 06:37:50.944813 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:50.944862 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:50.946618 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:50.947296 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:50.947304 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:50.948023 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:50.948762 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:50.948789 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:50.950136 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:50.950139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:50.951424 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:50.951436 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:50.952279 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:50.952284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:50.952458 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:50.953068 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:50.953095 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:50.955507 (+ 2412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:50.958513 (+ 3006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:50.958521 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:50.958537 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:50.959104 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:50.959464 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:50.959468 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:50.959471 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:50.959699 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:50.960332 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:50.960340 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:50.960768 (+ 428us) client_negotiation.cc:769] Sending connection context
0810 06:37:50.961054 (+ 286us) client_negotiation.cc:240] Negotiation successful
0810 06:37:50.961562 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":295,"threads_started":1}
I20260810 06:37:50.964565 7204 heartbeater.cc:347] Connected to a master server at 127.31.59.254:36905
I20260810 06:37:50.964911 7204 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:50.965708 7204 heartbeater.cc:511] Master 127.31.59.254:36905 requested a full tablet report, sending...
I20260810 06:37:50.967419 6658 ts_manager.cc:194] Registered new tserver with Master: ae286d56dd4d4e24afb779ea610d4303 (127.31.59.194:44857)
I20260810 06:37:50.968221 6658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:37833
WARNING: no policy specified for kudu/127.31.59.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
I20260810 06:37:50.997092 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36905
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:35573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:51.323762 7211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:51.324023 7211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:51.324141 7211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:51.331067 7211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:51.331178 7211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:51.331333 7211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:37:51.340514 7211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:35573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:51.341706 7211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:51.342973 7211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:51.355111 7221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:51.355087 7220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:51.355304 7224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:51.355928 7211 server_base.cc:1034] running on GCE node
I20260810 06:37:51.356418 7211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:51.357247 7211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:51.358418 7211 hybrid_clock.cc:648] HybridClock initialized: now 1786343871358372 us; error 42 us; skew 500 ppm
I20260810 06:37:51.358712 7211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:37:51 dist-test-slave-4tdj krb5kdc[6592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343871, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:51.361562 7211 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:37:51.362640 7211 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:51.362726 7211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:51.362975 7211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:51.365703 7211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "4c4a2f9032c940f185d48d87590e9467"
format_stamp: "Formatted at 2026-08-10 06:37:51 on dist-test-slave-4tdj"
server_key: "908b8b571ad8fe04d97efb2ee12e8df6"
server_key_iv: "51adb7395d401afae9063984be32e6af"
server_key_version: "encryptionkey@0"
I20260810 06:37:51.366467 7211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "4c4a2f9032c940f185d48d87590e9467"
format_stamp: "Formatted at 2026-08-10 06:37:51 on dist-test-slave-4tdj"
server_key: "908b8b571ad8fe04d97efb2ee12e8df6"
server_key_iv: "51adb7395d401afae9063984be32e6af"
server_key_version: "encryptionkey@0"
I20260810 06:37:51.371115 7211 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 06:37:51.374382 7238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:51.375619 7211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:37:51.375765 7211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "4c4a2f9032c940f185d48d87590e9467"
format_stamp: "Formatted at 2026-08-10 06:37:51 on dist-test-slave-4tdj"
server_key: "908b8b571ad8fe04d97efb2ee12e8df6"
server_key_iv: "51adb7395d401afae9063984be32e6af"
server_key_version: "encryptionkey@0"
I20260810 06:37:51.375964 7211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:51.398329 7211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:51.402943 7211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:51.403199 7211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:51.404122 7211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:51.405416 7211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:51.405504 7211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:51.405604 7211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:51.405670 7211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:51.435685 7211 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:42301
I20260810 06:37:51.435750 7444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:42301 every 8 connection(s)
I20260810 06:37:51.436546 7211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:37:51.446132 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 7211
I20260810 06:37:51.446329 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:37:51.446882 31983 external_mini_cluster.cc:1428] Setting key baa1a17d30f2d42ef354d104cb04a7dc
Aug 10 06:37:51 dist-test-slave-4tdj krb5kdc[6592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343871, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:37:51.457599 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:51.439055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:38657 (local address 127.31.59.254:36905)
0810 06:37:51.439242 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:51.439247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:51.440395 (+ 1148us) server_negotiation.cc:407] Connection header received
0810 06:37:51.442320 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:51.442326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:51.442407 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:51.442499 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:51.444090 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.444837 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.446603 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.447220 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.447362 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:51.451376 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:51.451382 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:51.451385 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:51.451428 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:51.454644 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:51.455332 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:51.455338 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:51.455342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:51.455485 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:51.455914 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:51.455919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:51.455923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:51.456281 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:51.456425 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:51.457068 (+ 643us) server_negotiation.cc:299] Negotiation successful
0810 06:37:51.457284 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:37:51.458266 7452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:51.439493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36905 (local address 127.31.59.195:38657)
0810 06:37:51.440225 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:51.440264 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:51.441989 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:51.442751 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:51.442760 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:51.443288 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:51.443896 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:51.443915 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.444989 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.444993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:51.446373 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:51.446390 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.447492 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.447503 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:51.447862 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:51.448660 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:51.448685 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:51.451167 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:51.454778 (+ 3611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:51.454785 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:51.454795 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:51.455212 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:51.455627 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:51.455631 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:51.455633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:51.455810 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:51.456411 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:51.456416 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:51.456726 (+ 310us) client_negotiation.cc:769] Sending connection context
0810 06:37:51.456942 (+ 216us) client_negotiation.cc:240] Negotiation successful
0810 06:37:51.457301 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":287,"threads_started":1}
I20260810 06:37:51.459580 7445 heartbeater.cc:347] Connected to a master server at 127.31.59.254:36905
I20260810 06:37:51.459849 7445 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:51.460446 7445 heartbeater.cc:511] Master 127.31.59.254:36905 requested a full tablet report, sending...
I20260810 06:37:51.461858 6658 ts_manager.cc:194] Registered new tserver with Master: 4c4a2f9032c940f185d48d87590e9467 (127.31.59.195:42301)
I20260810 06:37:51.462621 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:37:51.462783 6658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:38657
I20260810 06:37:51.480140 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:51.467236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35192 (local address 127.31.59.254:36905)
0810 06:37:51.467401 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:51.467406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:51.467965 (+ 559us) server_negotiation.cc:407] Connection header received
0810 06:37:51.468176 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:51.468181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:51.468255 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:51.468344 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:51.470245 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.471214 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.472551 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.472977 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.473074 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:51.475300 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:51.475305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:51.475308 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:51.475344 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:51.477730 (+ 2386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:51.478404 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:51.478408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:51.478410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:51.478556 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:51.478922 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:51.478925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:51.478927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:51.479220 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:51.479399 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:51.479710 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 06:37:51.479898 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:37:51.485083 6971 heartbeater.cc:503] Master 127.31.59.254:36905 was elected leader, sending a full tablet report...
I20260810 06:37:51.484023 6658 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:37:51.491022 6658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:37:51.514216 7470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:51.504060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:44857 (local address 127.0.0.1:57450)
0810 06:37:51.504588 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:51.504608 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:51.504727 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:51.505833 (+ 1106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:51.505839 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:51.505868 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:51.506345 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:51.506357 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.508635 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.508640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:51.511090 (+ 2450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:51.511106 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.513122 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.513129 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:51.513350 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:51.513406 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 06:37:51.513513 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 06:37:51.513715 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":33,"thread_start_us":246,"threads_started":1}
I20260810 06:37:51.514972 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:51.503099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35236 (local address 127.31.59.195:42301)
0810 06:37:51.503901 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:51.503910 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:51.503956 (+ 46us) server_negotiation.cc:407] Connection header received
0810 06:37:51.504064 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:51.504072 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:51.504340 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:51.504564 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:51.506073 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.507971 (+ 1898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.511317 (+ 3346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.512995 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.513157 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:51.513427 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:51.514396 (+ 969us) server_negotiation.cc:299] Negotiation successful
0810 06:37:51.514591 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":443,"threads_started":1}
I20260810 06:37:51.515241 7468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:51.502790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:42301 (local address 127.0.0.1:35236)
0810 06:37:51.503598 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:51.503646 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:51.503800 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:51.504921 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:51.504929 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:51.504964 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:51.505790 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:51.505810 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.508172 (+ 2362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.508179 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:51.511076 (+ 2897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:51.511097 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.513626 (+ 2529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.513634 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:51.513862 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:51.513904 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 06:37:51.514011 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 06:37:51.514172 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":28,"thread_start_us":446,"threads_started":1}
I20260810 06:37:51.516394 7471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:51.504285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57450 (local address 127.31.59.194:44857)
0810 06:37:51.505205 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:51.505227 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:51.505254 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:37:51.505357 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:51.505365 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:51.505581 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:51.506579 (+ 998us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:51.506821 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.508457 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.511322 (+ 2865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.512870 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.515554 (+ 2684us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:51.515778 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:51.515862 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 06:37:51.516048 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":776,"thread_start_us":629,"threads_started":1}
I20260810 06:37:51.520058 7327 tablet_service.cc:1467] Processing CreateTablet for tablet d294dcf32d13483eb2742a75060ceb21 (DEFAULT_TABLE table=test-table [id=ef8a76f25687424ea5ddd30595ad7bfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:51.520085 7100 tablet_service.cc:1467] Processing CreateTablet for tablet d294dcf32d13483eb2742a75060ceb21 (DEFAULT_TABLE table=test-table [id=ef8a76f25687424ea5ddd30595ad7bfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:51.521479 7327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d294dcf32d13483eb2742a75060ceb21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:51.521549 7100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d294dcf32d13483eb2742a75060ceb21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:51.526557 7470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:51.505262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:46435 (local address 127.0.0.1:40062)
0810 06:37:51.514539 (+ 9277us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:51.514558 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:51.514655 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:51.519660 (+ 5005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:51.519664 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:51.519690 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:51.519875 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:51.519887 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.521894 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.521900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:51.523864 (+ 1964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:51.523880 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.525764 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.525771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:51.525949 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:51.525985 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:37:51.526080 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 06:37:51.526226 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9181,"thread_start_us":13943,"threads_started":1}
I20260810 06:37:51.526821 7474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:51.505567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40062 (local address 127.31.59.193:46435)
0810 06:37:51.518533 (+ 12966us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:51.518542 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:51.518570 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:37:51.518678 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:51.518685 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:51.518901 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:51.520050 (+ 1149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:51.520296 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.521754 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.524087 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.525573 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.525739 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:51.525987 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:51.526145 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 06:37:51.526363 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12786,"thread_start_us":13808,"threads_started":1}
I20260810 06:37:51.530701 6869 tablet_service.cc:1467] Processing CreateTablet for tablet d294dcf32d13483eb2742a75060ceb21 (DEFAULT_TABLE table=test-table [id=ef8a76f25687424ea5ddd30595ad7bfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:51.532194 6869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d294dcf32d13483eb2742a75060ceb21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:51.535045 7476 tablet_bootstrap.cc:492] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467: Bootstrap starting.
I20260810 06:37:51.537202 7478 tablet_bootstrap.cc:492] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303: Bootstrap starting.
I20260810 06:37:51.539388 7476 tablet_bootstrap.cc:654] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:51.540726 7476 log.cc:824] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:51.540903 7478 tablet_bootstrap.cc:654] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:51.542037 7478 log.cc:824] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:51.543919 7476 tablet_bootstrap.cc:492] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467: No bootstrap required, opened a new log
I20260810 06:37:51.544217 7476 ts_tablet_manager.cc:1397] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 06:37:51.544838 7481 tablet_bootstrap.cc:492] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097: Bootstrap starting.
I20260810 06:37:51.545148 7478 tablet_bootstrap.cc:492] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303: No bootstrap required, opened a new log
I20260810 06:37:51.545447 7478 ts_tablet_manager.cc:1397] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260810 06:37:51.548231 7481 tablet_bootstrap.cc:654] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:51.549136 7481 log.cc:824] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:51.552230 7481 tablet_bootstrap.cc:492] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097: No bootstrap required, opened a new log
I20260810 06:37:51.552457 7481 ts_tablet_manager.cc:1397] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 06:37:51.556231 7476 raft_consensus.cc:348] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4a2f9032c940f185d48d87590e9467" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42301 } } peers { permanent_uuid: "ae286d56dd4d4e24afb779ea610d4303" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 44857 } } peers { permanent_uuid: "e901ca39bfc148169a17a83d8ab93097" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46435 } }
I20260810 06:37:51.556231 7478 raft_consensus.cc:348] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4a2f9032c940f185d48d87590e9467" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42301 } } peers { permanent_uuid: "ae286d56dd4d4e24afb779ea610d4303" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 44857 } } peers { permanent_uuid: "e901ca39bfc148169a17a83d8ab93097" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46435 } }
I20260810 06:37:51.556697 7478 raft_consensus.cc:374] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:51.556697 7476 raft_consensus.cc:374] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:51.556811 7476 raft_consensus.cc:729] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c4a2f9032c940f185d48d87590e9467, State: Initialized, Role: FOLLOWER
I20260810 06:37:51.556811 7478 raft_consensus.cc:729] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae286d56dd4d4e24afb779ea610d4303, State: Initialized, Role: FOLLOWER
I20260810 06:37:51.557457 7478 consensus_queue.cc:260] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4a2f9032c940f185d48d87590e9467" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42301 } } peers { permanent_uuid: "ae286d56dd4d4e24afb779ea610d4303" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 44857 } } peers { permanent_uuid: "e901ca39bfc148169a17a83d8ab93097" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46435 } }
I20260810 06:37:51.557456 7476 consensus_queue.cc:260] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4a2f9032c940f185d48d87590e9467" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42301 } } peers { permanent_uuid: "ae286d56dd4d4e24afb779ea610d4303" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 44857 } } peers { permanent_uuid: "e901ca39bfc148169a17a83d8ab93097" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46435 } }
I20260810 06:37:51.559024 7445 heartbeater.cc:503] Master 127.31.59.254:36905 was elected leader, sending a full tablet report...
I20260810 06:37:51.559834 7476 ts_tablet_manager.cc:1428] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467: Time spent starting tablet: real 0.015s user 0.008s sys 0.007s
I20260810 06:37:51.559880 7481 raft_consensus.cc:348] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4a2f9032c940f185d48d87590e9467" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42301 } } peers { permanent_uuid: "ae286d56dd4d4e24afb779ea610d4303" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 44857 } } peers { permanent_uuid: "e901ca39bfc148169a17a83d8ab93097" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46435 } }
I20260810 06:37:51.560230 7481 raft_consensus.cc:374] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:51.560322 7481 raft_consensus.cc:729] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e901ca39bfc148169a17a83d8ab93097, State: Initialized, Role: FOLLOWER
I20260810 06:37:51.560921 7481 consensus_queue.cc:260] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4a2f9032c940f185d48d87590e9467" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42301 } } peers { permanent_uuid: "ae286d56dd4d4e24afb779ea610d4303" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 44857 } } peers { permanent_uuid: "e901ca39bfc148169a17a83d8ab93097" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46435 } }
I20260810 06:37:51.562815 7481 ts_tablet_manager.cc:1428] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097: Time spent starting tablet: real 0.010s user 0.011s sys 0.001s
I20260810 06:37:51.565886 7204 heartbeater.cc:503] Master 127.31.59.254:36905 was elected leader, sending a full tablet report...
I20260810 06:37:51.566843 7478 ts_tablet_manager.cc:1428] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303: Time spent starting tablet: real 0.021s user 0.008s sys 0.013s
I20260810 06:37:51.573750 7485 raft_consensus.cc:482] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:37:51.573961 7485 raft_consensus.cc:504] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4a2f9032c940f185d48d87590e9467" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42301 } } peers { permanent_uuid: "ae286d56dd4d4e24afb779ea610d4303" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 44857 } } peers { permanent_uuid: "e901ca39bfc148169a17a83d8ab93097" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46435 } }
I20260810 06:37:51.575440 7485 leader_election.cc:288] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c4a2f9032c940f185d48d87590e9467 (127.31.59.195:42301), e901ca39bfc148169a17a83d8ab93097 (127.31.59.193:46435)
I20260810 06:37:51.582653 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:51.575742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:45475 (local address 127.31.59.195:42301)
0810 06:37:51.575943 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:51.575949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:51.576706 (+ 757us) server_negotiation.cc:407] Connection header received
0810 06:37:51.576871 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:51.576875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:51.576928 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:51.576998 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:51.577548 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.578240 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.580970 (+ 2730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.581757 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.581875 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:51.581925 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:51.582283 (+ 358us) server_negotiation.cc:299] Negotiation successful
0810 06:37:51.582412 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:37:51.582788 7474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:51.575914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:58361 (local address 127.31.59.193:46435)
0810 06:37:51.576097 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:51.576103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:51.576744 (+ 641us) server_negotiation.cc:407] Connection header received
0810 06:37:51.576892 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:51.576896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:51.576947 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:51.577023 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:51.577636 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.578471 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.581041 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.581756 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.581892 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:51.581962 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:51.582447 (+ 485us) server_negotiation.cc:299] Negotiation successful
0810 06:37:51.582582 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:37:51.582840 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:51.575916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:46435 (local address 127.31.59.194:58361)
0810 06:37:51.576649 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:51.576663 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:51.576779 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:51.577040 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:51.577044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:51.577067 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:51.577492 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:51.577508 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.578640 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.578645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:51.580839 (+ 2194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:51.580854 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.582120 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.582126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:51.582280 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:51.582334 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 06:37:51.582407 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 06:37:51.582505 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":5,"thread_start_us":403,"threads_started":1}
I20260810 06:37:51.583189 7487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:51.575815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:42301 (local address 127.31.59.194:45475)
0810 06:37:51.576608 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:51.576623 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:51.576752 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:51.577022 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:51.577027 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:51.577050 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:51.577406 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:51.577416 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.578378 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.578383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:51.580846 (+ 2463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:51.580855 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.581953 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.581959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:51.582134 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:51.582174 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 06:37:51.582270 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 06:37:51.582418 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"mutex_wait_us":80,"thread_start_us":415,"threads_started":1}
I20260810 06:37:51.583662 6900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d294dcf32d13483eb2742a75060ceb21" candidate_uuid: "ae286d56dd4d4e24afb779ea610d4303" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e901ca39bfc148169a17a83d8ab93097" is_pre_election: true
I20260810 06:37:51.584089 7364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d294dcf32d13483eb2742a75060ceb21" candidate_uuid: "ae286d56dd4d4e24afb779ea610d4303" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c4a2f9032c940f185d48d87590e9467" is_pre_election: true
I20260810 06:37:51.584228 6900 raft_consensus.cc:2454] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae286d56dd4d4e24afb779ea610d4303 in term 0.
I20260810 06:37:51.584607 7364 raft_consensus.cc:2454] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae286d56dd4d4e24afb779ea610d4303 in term 0.
I20260810 06:37:51.584947 7012 leader_election.cc:302] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae286d56dd4d4e24afb779ea610d4303, e901ca39bfc148169a17a83d8ab93097; no voters:
I20260810 06:37:51.585388 7485 raft_consensus.cc:2781] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:37:51.585512 7485 raft_consensus.cc:482] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:37:51.585611 7485 raft_consensus.cc:3037] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:51.587725 7485 raft_consensus.cc:504] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4a2f9032c940f185d48d87590e9467" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42301 } } peers { permanent_uuid: "ae286d56dd4d4e24afb779ea610d4303" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 44857 } } peers { permanent_uuid: "e901ca39bfc148169a17a83d8ab93097" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46435 } }
I20260810 06:37:51.588274 7485 leader_election.cc:288] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [CANDIDATE]: Term 1 election: Requested vote from peers 4c4a2f9032c940f185d48d87590e9467 (127.31.59.195:42301), e901ca39bfc148169a17a83d8ab93097 (127.31.59.193:46435)
I20260810 06:37:51.588694 7364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d294dcf32d13483eb2742a75060ceb21" candidate_uuid: "ae286d56dd4d4e24afb779ea610d4303" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c4a2f9032c940f185d48d87590e9467"
I20260810 06:37:51.588760 6900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d294dcf32d13483eb2742a75060ceb21" candidate_uuid: "ae286d56dd4d4e24afb779ea610d4303" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e901ca39bfc148169a17a83d8ab93097"
I20260810 06:37:51.588894 7364 raft_consensus.cc:3037] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:51.588917 6900 raft_consensus.cc:3037] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:51.591948 7364 raft_consensus.cc:2454] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae286d56dd4d4e24afb779ea610d4303 in term 1.
I20260810 06:37:51.591948 6900 raft_consensus.cc:2454] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae286d56dd4d4e24afb779ea610d4303 in term 1.
I20260810 06:37:51.592329 7012 leader_election.cc:302] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae286d56dd4d4e24afb779ea610d4303, e901ca39bfc148169a17a83d8ab93097; no voters:
I20260810 06:37:51.592592 7485 raft_consensus.cc:2781] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:51.592984 7485 raft_consensus.cc:686] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [term 1 LEADER]: Becoming Leader. State: Replica: ae286d56dd4d4e24afb779ea610d4303, State: Running, Role: LEADER
I20260810 06:37:51.593405 7485 consensus_queue.cc:237] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4a2f9032c940f185d48d87590e9467" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42301 } } peers { permanent_uuid: "ae286d56dd4d4e24afb779ea610d4303" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 44857 } } peers { permanent_uuid: "e901ca39bfc148169a17a83d8ab93097" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46435 } }
I20260810 06:37:51.597489 6656 catalog_manager.cc:5495] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae286d56dd4d4e24afb779ea610d4303 (127.31.59.194). New cstate: current_term: 1 leader_uuid: "ae286d56dd4d4e24afb779ea610d4303" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4a2f9032c940f185d48d87590e9467" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae286d56dd4d4e24afb779ea610d4303" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 44857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e901ca39bfc148169a17a83d8ab93097" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46435 } health_report { overall_health: UNKNOWN } } }
I20260810 06:37:51.626073 7471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:51.620873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57462 (local address 127.31.59.194:44857)
0810 06:37:51.621072 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:51.621077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:51.621097 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:37:51.621196 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:51.621200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:51.621274 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:51.621404 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:37:51.621818 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.622632 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.624029 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.624428 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.624535 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:51.624996 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:37:51.625489 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:37:51.625637 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:51.625733 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 06:37:51.625810 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:37:51.634222 7364 raft_consensus.cc:1261] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467 [term 1 FOLLOWER]: Refusing update from remote peer ae286d56dd4d4e24afb779ea610d4303: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:51.634222 6900 raft_consensus.cc:1261] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097 [term 1 FOLLOWER]: Refusing update from remote peer ae286d56dd4d4e24afb779ea610d4303: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:51.635329 7489 consensus_queue.cc:1035] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e901ca39bfc148169a17a83d8ab93097" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:51.635656 7485 consensus_queue.cc:1035] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c4a2f9032c940f185d48d87590e9467" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:51.647480 7493 mvcc.cc:205] Tried to move back new op lower bound from 7316864498203103232 to 7316864498050064384. Current Snapshot: MvccSnapshot[applied={T|T < 7316864498203103232}]
I20260810 06:37:51.654647 7495 mvcc.cc:205] Tried to move back new op lower bound from 7316864498203103232 to 7316864498050064384. Current Snapshot: MvccSnapshot[applied={T|T < 7316864498203103232}]
I20260810 06:37:51.658846 6658 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:37:51.659137 6658 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:37:51.662622 6658 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0021da411821463c95d67e7203641266: Sending DeleteTablet for 3 replicas of tablet d294dcf32d13483eb2742a75060ceb21
I20260810 06:37:51.663642 7327 tablet_service.cc:1514] Processing DeleteTablet for tablet d294dcf32d13483eb2742a75060ceb21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:51 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:35236
I20260810 06:37:51.663682 7100 tablet_service.cc:1514] Processing DeleteTablet for tablet d294dcf32d13483eb2742a75060ceb21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:51 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:57450
I20260810 06:37:51.664150 6869 tablet_service.cc:1514] Processing DeleteTablet for tablet d294dcf32d13483eb2742a75060ceb21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:51 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:40062
I20260810 06:37:51.665707 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 6755
I20260810 06:37:51.677590 7513 tablet_replica.cc:330] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303: stopping tablet replica
I20260810 06:37:51.678009 7513 raft_consensus.cc:2229] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:37:51.678712 7513 raft_consensus.cc:2258] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:51.679558 7516 tablet_replica.cc:330] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097: stopping tablet replica
I20260810 06:37:51.679940 7516 raft_consensus.cc:2229] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:37:51.680569 7516 raft_consensus.cc:2258] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:51.683542 7516 ts_tablet_manager.cc:1905] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 06:37:51 dist-test-slave-4tdj krb5kdc[6592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:37:51.690172 7513 ts_tablet_manager.cc:1905] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:51.690852 7512 tablet_replica.cc:330] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467: stopping tablet replica
I20260810 06:37:51.691251 7512 raft_consensus.cc:2229] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:37:51.691726 7512 raft_consensus.cc:2258] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:51.691933 7516 ts_tablet_manager.cc:1918] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:37:51.692157 7516 log.cc:1196] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/d294dcf32d13483eb2742a75060ceb21
I20260810 06:37:51.692756 7516 ts_tablet_manager.cc:1939] T d294dcf32d13483eb2742a75060ceb21 P e901ca39bfc148169a17a83d8ab93097: Deleting consensus metadata
I20260810 06:37:51.694761 6631 catalog_manager.cc:4841] TS e901ca39bfc148169a17a83d8ab93097 (127.31.59.193:46435): tablet d294dcf32d13483eb2742a75060ceb21 (table test-table [id=ef8a76f25687424ea5ddd30595ad7bfc]) successfully deleted
I20260810 06:37:51.696887 7512 ts_tablet_manager.cc:1905] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:51.697923 7474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:51.668520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40070 (local address 127.31.59.193:46435)
0810 06:37:51.668700 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:51.668706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:51.673231 (+ 4525us) server_negotiation.cc:407] Connection header received
0810 06:37:51.673416 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:51.673422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:51.673513 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:51.673622 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:51.680036 (+ 6414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.681317 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.683815 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.684564 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.684661 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:51.690086 (+ 5425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:51.690099 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:51.690115 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:51.690155 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:51.693866 (+ 3711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:51.694767 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:51.694775 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:51.694778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:51.694925 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:51.695365 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:51.695369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:51.695373 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:51.695815 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:51.696120 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:51.697355 (+ 1235us) server_negotiation.cc:299] Negotiation successful
0810 06:37:51.697585 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:37:51.698529 6963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:51.699378 7513 ts_tablet_manager.cc:1918] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:37:51.699548 7513 log.cc:1196] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/d294dcf32d13483eb2742a75060ceb21
I20260810 06:37:51.700181 7513 ts_tablet_manager.cc:1939] T d294dcf32d13483eb2742a75060ceb21 P ae286d56dd4d4e24afb779ea610d4303: Deleting consensus metadata
I20260810 06:37:51.701912 6630 catalog_manager.cc:4841] TS ae286d56dd4d4e24afb779ea610d4303 (127.31.59.194:44857): tablet d294dcf32d13483eb2742a75060ceb21 (table test-table [id=ef8a76f25687424ea5ddd30595ad7bfc]) successfully deleted
I20260810 06:37:51.704928 7512 ts_tablet_manager.cc:1918] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:37:51.705153 7512 log.cc:1196] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/d294dcf32d13483eb2742a75060ceb21
I20260810 06:37:51.705770 7512 ts_tablet_manager.cc:1939] T d294dcf32d13483eb2742a75060ceb21 P 4c4a2f9032c940f185d48d87590e9467: Deleting consensus metadata
I20260810 06:37:51.707110 6635 catalog_manager.cc:4841] TS 4c4a2f9032c940f185d48d87590e9467 (127.31.59.195:42301): tablet d294dcf32d13483eb2742a75060ceb21 (table test-table [id=ef8a76f25687424ea5ddd30595ad7bfc]) successfully deleted
I20260810 06:37:51.818362 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 6755
I20260810 06:37:51.837643 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 6982
Aug 10 06:37:51 dist-test-slave-4tdj krb5kdc[6592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.194@KRBTEST.COM
I20260810 06:37:51.854585 7471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:51.838401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57464 (local address 127.31.59.194:44857)
0810 06:37:51.838596 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:51.838602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:51.838691 (+ 89us) server_negotiation.cc:407] Connection header received
0810 06:37:51.838875 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:51.838880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:51.838966 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:51.839066 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:51.840299 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.841141 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.842538 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:51.843237 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:51.843387 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:51.847383 (+ 3996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:51.847407 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:51.847429 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:51.847495 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:51.851592 (+ 4097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:51.852272 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:51.852283 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:51.852287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:51.852476 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:51.852814 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:51.852820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:51.852824 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:51.853314 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:51.853662 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:51.853846 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 06:37:51.854103 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:37:51.855320 7189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:51.974773 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 6982
I20260810 06:37:51.998417 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 7211
Aug 10 06:37:52 dist-test-slave-4tdj krb5kdc[6592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.195@KRBTEST.COM
I20260810 06:37:52.013124 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:51.999150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35252 (local address 127.31.59.195:42301)
0810 06:37:51.999334 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:51.999338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:51.999358 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:37:51.999504 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:51.999508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:51.999574 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:51.999651 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:52.000891 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:52.001687 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:52.003010 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:52.003420 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:52.003533 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:52.007609 (+ 4076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:52.007627 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:52.007643 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:52.007688 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:52.010689 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:52.011286 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:52.011293 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:52.011296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:52.011448 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:52.011817 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:52.011821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:52.011823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:52.012195 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:52.012410 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:52.012638 (+ 228us) server_negotiation.cc:299] Negotiation successful
0810 06:37:52.012799 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:37:52.013723 7430 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:52.099804 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 7211
I20260810 06:37:52.121595 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 6612
I20260810 06:37:52.122762 6703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:52.228364 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 6612
2026-08-10T06:37:52Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4158 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T06:37:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:37:52Z Disabled control of system clock
I20260810 06:37:52.301965 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:39821
--webserver_interface=127.31.59.254
--webserver_port=0
--builtin_ntp_servers=127.31.59.212:36391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:39821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 06:37:52.613751 7528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:52.614017 7528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:52.614144 7528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:52.621264 7528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:37:52.621358 7528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:52.621424 7528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:52.621500 7528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:37:52.621556 7528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:37:52.630492 7528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:39821
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:39821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:52.631708 7528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:52.633437 7528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:52.644586 7539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:52.644523 7537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:52.644614 7536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:52.645236 7528 server_base.cc:1034] running on GCE node
I20260810 06:37:52.645998 7528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:52.647370 7528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:52.648590 7528 hybrid_clock.cc:648] HybridClock initialized: now 1786343872648537 us; error 40 us; skew 500 ppm
I20260810 06:37:52.648931 7528 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:37:52.651072 7528 webserver.cc:460] Webserver started at http://127.31.59.254:39595/ using document root <none> and password file <none>
I20260810 06:37:52.651707 7528 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:52.651793 7528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:52.652101 7528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:52.654889 7528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "d7a46e82d0614ef783157ede220e9540"
format_stamp: "Formatted at 2026-08-10 06:37:52 on dist-test-slave-4tdj"
server_key: "925fa9fd6e7aca3b221df5b75cc56ef8"
server_key_iv: "c67fbc5dab18dc0e5b9c484c31a81e61"
server_key_version: "encryptionkey@0"
I20260810 06:37:52.655649 7528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "d7a46e82d0614ef783157ede220e9540"
format_stamp: "Formatted at 2026-08-10 06:37:52 on dist-test-slave-4tdj"
server_key: "925fa9fd6e7aca3b221df5b75cc56ef8"
server_key_iv: "c67fbc5dab18dc0e5b9c484c31a81e61"
server_key_version: "encryptionkey@0"
I20260810 06:37:52.660426 7528 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:37:52.663605 7546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:52.664824 7528 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:37:52.664970 7528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "d7a46e82d0614ef783157ede220e9540"
format_stamp: "Formatted at 2026-08-10 06:37:52 on dist-test-slave-4tdj"
server_key: "925fa9fd6e7aca3b221df5b75cc56ef8"
server_key_iv: "c67fbc5dab18dc0e5b9c484c31a81e61"
server_key_version: "encryptionkey@0"
I20260810 06:37:52.665162 7528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:52.670548 7528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:52.671234 7528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:52.671514 7528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:52.688675 7528 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:39821
I20260810 06:37:52.688715 7625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:39821 every 8 connection(s)
I20260810 06:37:52.690330 7528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:37:52.697945 7626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:52.698146 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 7528
I20260810 06:37:52.698390 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:37:52.699249 31983 external_mini_cluster.cc:1428] Setting key b87583d74450e0110837df9d76ef44d2
I20260810 06:37:52.708491 7631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:52.701499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35780 (local address 127.31.59.254:39821)
0810 06:37:52.702721 (+ 1222us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:52.702735 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:52.702856 (+ 121us) server_negotiation.cc:407] Connection header received
0810 06:37:52.704609 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:52.704638 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:52.705195 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:52.705655 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:52.706611 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:52.706619 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:52.706633 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:52.706649 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:52.706723 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:52.706744 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:52.706953 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:52.707090 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 06:37:52.707341 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":391,"threads_started":1}
I20260810 06:37:52.712240 7626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540: Bootstrap starting.
I20260810 06:37:52.715095 7626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:52.716120 7626 log.cc:824] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:52.718775 7626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540: No bootstrap required, opened a new log
I20260810 06:37:52.725747 7626 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a46e82d0614ef783157ede220e9540" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 39821 } }
I20260810 06:37:52.726068 7626 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:52.726164 7626 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a46e82d0614ef783157ede220e9540, State: Initialized, Role: FOLLOWER
I20260810 06:37:52.726763 7626 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a46e82d0614ef783157ede220e9540" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 39821 } }
I20260810 06:37:52.726974 7626 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:37:52.727102 7626 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:37:52.727248 7626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:52.729250 7626 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a46e82d0614ef783157ede220e9540" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 39821 } }
I20260810 06:37:52.729715 7626 leader_election.cc:302] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7a46e82d0614ef783157ede220e9540; no voters:
I20260810 06:37:52.730336 7626 leader_election.cc:288] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:37:52.730507 7634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:52.731266 7634 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540 [term 1 LEADER]: Becoming Leader. State: Replica: d7a46e82d0614ef783157ede220e9540, State: Running, Role: LEADER
I20260810 06:37:52.731765 7634 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a46e82d0614ef783157ede220e9540" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 39821 } }
I20260810 06:37:52.732301 7626 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:37:52.734366 7636 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7a46e82d0614ef783157ede220e9540. Latest consensus state: current_term: 1 leader_uuid: "d7a46e82d0614ef783157ede220e9540" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a46e82d0614ef783157ede220e9540" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 39821 } } }
I20260810 06:37:52.734761 7636 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:52.734896 7635 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7a46e82d0614ef783157ede220e9540" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a46e82d0614ef783157ede220e9540" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 39821 } } }
I20260810 06:37:52.735217 7635 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:52.736721 7649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:37:52.741828 7649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:37:52.750937 7649 catalog_manager.cc:1347] Generated new cluster ID: a5b8a339c0ae41c2996b6f2538fd29a3
I20260810 06:37:52.751044 7649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:37:52.765744 7649 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:37:52.766099 7649 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:37:52.776906 7649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540: Generated new TSK 0
I20260810 06:37:52.777622 7649 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:37:52.783753 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:39821
--builtin_ntp_servers=127.31.59.212:36391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 06:37:53.136503 7664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:53.136766 7664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:53.136896 7664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:53.143867 7664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:53.143955 7664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:53.144102 7664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:37:53.152935 7664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:39821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:53.154155 7664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:53.155921 7664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:53.168331 7674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:53.168341 7675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:53.168388 7677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:53.168943 7664 server_base.cc:1034] running on GCE node
I20260810 06:37:53.169584 7664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:53.170475 7664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:53.171693 7664 hybrid_clock.cc:648] HybridClock initialized: now 1786343873171628 us; error 53 us; skew 500 ppm
I20260810 06:37:53.172003 7664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:37:53.174163 7664 webserver.cc:460] Webserver started at http://127.31.59.193:33605/ using document root <none> and password file <none>
I20260810 06:37:53.174837 7664 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:53.174927 7664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:53.175209 7664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:53.177946 7664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "b8b5766828b446ce85c2b4152478c32c"
format_stamp: "Formatted at 2026-08-10 06:37:53 on dist-test-slave-4tdj"
server_key: "90743af86c198603cab0d38c46b8a0b9"
server_key_iv: "eef0563e00a8a80e874481a6debd0cab"
server_key_version: "encryptionkey@0"
I20260810 06:37:53.178725 7664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "b8b5766828b446ce85c2b4152478c32c"
format_stamp: "Formatted at 2026-08-10 06:37:53 on dist-test-slave-4tdj"
server_key: "90743af86c198603cab0d38c46b8a0b9"
server_key_iv: "eef0563e00a8a80e874481a6debd0cab"
server_key_version: "encryptionkey@0"
I20260810 06:37:53.183605 7664 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:37:53.186937 7690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:53.188200 7664 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:37:53.188349 7664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "b8b5766828b446ce85c2b4152478c32c"
format_stamp: "Formatted at 2026-08-10 06:37:53 on dist-test-slave-4tdj"
server_key: "90743af86c198603cab0d38c46b8a0b9"
server_key_iv: "eef0563e00a8a80e874481a6debd0cab"
server_key_version: "encryptionkey@0"
I20260810 06:37:53.188640 7664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:53.194604 7664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:53.195394 7664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:53.195678 7664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:53.196606 7664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:53.198046 7664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:53.198156 7664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:53.198275 7664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:53.198371 7664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260810 06:37:53.228356 7664 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:44167
I20260810 06:37:53.228411 7897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:44167 every 8 connection(s)
I20260810 06:37:53.229661 7664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:37:53.237437 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 7664
I20260810 06:37:53.237600 7902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:53.232101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:42795 (local address 127.31.59.254:39821)
0810 06:37:53.232510 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:53.232514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:53.233715 (+ 1201us) server_negotiation.cc:407] Connection header received
0810 06:37:53.235413 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:53.235418 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:53.235487 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:53.235583 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:53.236849 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:53.236853 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:53.236856 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:53.236872 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:53.236918 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:53.236930 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:53.237050 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:53.237247 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 06:37:53.237315 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":195,"threads_started":1}
I20260810 06:37:53.237638 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:37:53.238241 31983 external_mini_cluster.cc:1428] Setting key ba5e10d24633ac29e09af9a66c928a93
I20260810 06:37:53.238528 7903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:53.232631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:39821 (local address 127.31.59.193:42795)
0810 06:37:53.233572 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:53.233608 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:53.235159 (+ 1551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:53.235811 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:53.235819 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:53.235904 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:53.236570 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:37:53.236573 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:53.236609 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:37:53.236690 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:53.237023 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:53.237034 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:53.237064 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 06:37:53.237313 (+ 249us) client_negotiation.cc:240] Negotiation successful
0810 06:37:53.237557 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":300,"threads_started":1}
I20260810 06:37:53.239939 7898 heartbeater.cc:347] Connected to a master server at 127.31.59.254:39821
I20260810 06:37:53.240259 7898 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:53.240959 7898 heartbeater.cc:511] Master 127.31.59.254:39821 requested a full tablet report, sending...
I20260810 06:37:53.241315 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:39821
--builtin_ntp_servers=127.31.59.212:36391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 06:37:53.243140 7569 ts_manager.cc:194] Registered new tserver with Master: b8b5766828b446ce85c2b4152478c32c (127.31.59.193:44167)
W20260810 06:37:53.588176 7904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:53.588549 7904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:53.588704 7904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:53.600406 7904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:53.600528 7904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:53.600696 7904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:37:53.615399 7904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:39821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:53.617081 7904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:53.619503 7904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:53.634141 7913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:53.635516 7914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:53.640076 7916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:53.641036 7904 server_base.cc:1034] running on GCE node
I20260810 06:37:53.641664 7904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:53.642637 7904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:53.643831 7904 hybrid_clock.cc:648] HybridClock initialized: now 1786343873643770 us; error 58 us; skew 500 ppm
I20260810 06:37:53.644136 7904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:37:53.646405 7904 webserver.cc:460] Webserver started at http://127.31.59.194:37971/ using document root <none> and password file <none>
I20260810 06:37:53.647064 7904 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:53.647164 7904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:53.647485 7904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:53.650369 7904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "2c84b32290b1403fac2a3c2ef153374a"
format_stamp: "Formatted at 2026-08-10 06:37:53 on dist-test-slave-4tdj"
server_key: "14036e66ee55117fc2d4b9322e5e5b48"
server_key_iv: "c16259af6d8c4ddd3c33bcefce0364cd"
server_key_version: "encryptionkey@0"
I20260810 06:37:53.651136 7904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "2c84b32290b1403fac2a3c2ef153374a"
format_stamp: "Formatted at 2026-08-10 06:37:53 on dist-test-slave-4tdj"
server_key: "14036e66ee55117fc2d4b9322e5e5b48"
server_key_iv: "c16259af6d8c4ddd3c33bcefce0364cd"
server_key_version: "encryptionkey@0"
I20260810 06:37:53.656075 7904 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:37:53.659438 7931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:53.660764 7904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:37:53.660905 7904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "2c84b32290b1403fac2a3c2ef153374a"
format_stamp: "Formatted at 2026-08-10 06:37:53 on dist-test-slave-4tdj"
server_key: "14036e66ee55117fc2d4b9322e5e5b48"
server_key_iv: "c16259af6d8c4ddd3c33bcefce0364cd"
server_key_version: "encryptionkey@0"
I20260810 06:37:53.661098 7904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:53.666467 7904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:53.667181 7904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:53.667409 7904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:53.668334 7904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:53.669783 7904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:53.669878 7904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:53.669976 7904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:53.670044 7904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:53.705601 7904 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:38027
I20260810 06:37:53.705674 8144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:38027 every 8 connection(s)
I20260810 06:37:53.707119 7904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:37:53.713344 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 7904
I20260810 06:37:53.713541 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:37:53.714097 31983 external_mini_cluster.cc:1428] Setting key 3e29444cc47f3b55e8fe931804747162
I20260810 06:37:53.715649 7902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:53.709856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:51637 (local address 127.31.59.254:39821)
0810 06:37:53.710048 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:53.710055 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:53.711655 (+ 1600us) server_negotiation.cc:407] Connection header received
0810 06:37:53.713341 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:53.713346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:53.713408 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:53.713503 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:53.714911 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:53.714916 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:53.714921 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:53.714940 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:53.714969 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:53.714980 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:53.715103 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:53.715346 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 06:37:53.715413 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 06:37:53.716804 8148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:53.710511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:39821 (local address 127.31.59.194:51637)
0810 06:37:53.711500 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:53.711555 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:53.713073 (+ 1518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:53.713832 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:53.713840 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:53.713922 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:53.714627 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:37:53.714630 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:53.714666 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:37:53.714747 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:53.715109 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:53.715122 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:53.715132 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 06:37:53.715438 (+ 306us) client_negotiation.cc:240] Negotiation successful
0810 06:37:53.715692 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":300,"threads_started":1}
I20260810 06:37:53.717301 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:39821
--builtin_ntp_servers=127.31.59.212:36391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 06:37:53.719630 8145 heartbeater.cc:347] Connected to a master server at 127.31.59.254:39821
I20260810 06:37:53.719919 8145 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:53.720553 8145 heartbeater.cc:511] Master 127.31.59.254:39821 requested a full tablet report, sending...
I20260810 06:37:53.722126 7569 ts_manager.cc:194] Registered new tserver with Master: 2c84b32290b1403fac2a3c2ef153374a (127.31.59.194:38027)
W20260810 06:37:54.033010 8151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:54.033288 8151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:54.033416 8151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:54.040503 8151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:54.040611 8151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:54.040758 8151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:37:54.049440 8151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:39821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:54.050654 8151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:54.052371 8151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:54.064993 8159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:54.065387 8157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:54.065608 8164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:54.065986 8151 server_base.cc:1034] running on GCE node
I20260810 06:37:54.066607 8151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:54.067474 8151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:54.068665 8151 hybrid_clock.cc:648] HybridClock initialized: now 1786343874068606 us; error 51 us; skew 500 ppm
I20260810 06:37:54.068987 8151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:37:54.071211 8151 webserver.cc:460] Webserver started at http://127.31.59.195:39389/ using document root <none> and password file <none>
I20260810 06:37:54.071826 8151 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:54.071925 8151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:54.072234 8151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:54.074896 8151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "8b9c6df9b1de4b5eb83f21c1afff85d6"
format_stamp: "Formatted at 2026-08-10 06:37:54 on dist-test-slave-4tdj"
server_key: "48061a4f8070a76541773fa3cdba3407"
server_key_iv: "a1327db0377b21a18682c1b63aa9056c"
server_key_version: "encryptionkey@0"
I20260810 06:37:54.075655 8151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "8b9c6df9b1de4b5eb83f21c1afff85d6"
format_stamp: "Formatted at 2026-08-10 06:37:54 on dist-test-slave-4tdj"
server_key: "48061a4f8070a76541773fa3cdba3407"
server_key_iv: "a1327db0377b21a18682c1b63aa9056c"
server_key_version: "encryptionkey@0"
I20260810 06:37:54.080344 8151 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 06:37:54.083465 8171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:54.084636 8151 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:37:54.084780 8151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "8b9c6df9b1de4b5eb83f21c1afff85d6"
format_stamp: "Formatted at 2026-08-10 06:37:54 on dist-test-slave-4tdj"
server_key: "48061a4f8070a76541773fa3cdba3407"
server_key_iv: "a1327db0377b21a18682c1b63aa9056c"
server_key_version: "encryptionkey@0"
I20260810 06:37:54.084959 8151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:54.090083 8151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:54.090794 8151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:54.091037 8151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:54.091841 8151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:54.093199 8151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:54.093297 8151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:54.093397 8151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:54.093475 8151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:54.123068 8151 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:45175
I20260810 06:37:54.123144 8361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:45175 every 8 connection(s)
I20260810 06:37:54.124338 8151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:37:54.127696 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 8151
I20260810 06:37:54.127902 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:37:54.128470 31983 external_mini_cluster.cc:1428] Setting key 622c3065aa5a8d4f6b5d1589e7901e2d
I20260810 06:37:54.132434 7902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.126964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:55073 (local address 127.31.59.254:39821)
0810 06:37:54.127211 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:54.127218 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:54.128566 (+ 1348us) server_negotiation.cc:407] Connection header received
0810 06:37:54.130341 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:54.130346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:54.130407 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:54.130491 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:54.131707 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:54.131712 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:54.131717 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:54.131734 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:54.131764 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:54.131775 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:54.131870 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:54.132086 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 06:37:54.132149 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260810 06:37:54.133260 8367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.127398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:39821 (local address 127.31.59.195:55073)
0810 06:37:54.128391 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:54.128430 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:54.130070 (+ 1640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:54.130726 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:54.130734 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:54.130820 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:54.131425 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:37:54.131428 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:54.131463 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:37:54.131554 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:54.131859 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:54.131869 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:54.131877 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:37:54.132120 (+ 243us) client_negotiation.cc:240] Negotiation successful
0810 06:37:54.132361 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":329,"threads_started":1}
I20260810 06:37:54.134460 8363 heartbeater.cc:347] Connected to a master server at 127.31.59.254:39821
I20260810 06:37:54.134748 8363 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:54.135355 8363 heartbeater.cc:511] Master 127.31.59.254:39821 requested a full tablet report, sending...
I20260810 06:37:54.136647 7569 ts_manager.cc:194] Registered new tserver with Master: 8b9c6df9b1de4b5eb83f21c1afff85d6 (127.31.59.195:45175)
I20260810 06:37:54.143635 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:37:54.150710 7902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.148091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35784 (local address 127.31.59.254:39821)
0810 06:37:54.148264 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:54.148268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:54.148794 (+ 526us) server_negotiation.cc:407] Connection header received
0810 06:37:54.148971 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:54.148977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:54.149040 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:54.149123 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:54.150020 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:54.150027 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:54.150032 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:54.150051 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:54.150085 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:54.150098 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:54.150214 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:54.150353 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 06:37:54.150419 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:37:54.154644 7569 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35784:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:37:54.161687 7569 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:37:54.189625 8384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.174563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:38027 (local address 127.0.0.1:40300)
0810 06:37:54.184733 (+ 10170us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:54.184774 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:54.184906 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:54.186595 (+ 1689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:54.186605 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:54.186697 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:54.187951 (+ 1254us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:37:54.187955 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:54.187975 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:37:54.188024 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:54.188559 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:54.188571 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:54.188580 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 06:37:54.188891 (+ 311us) client_negotiation.cc:240] Negotiation successful
0810 06:37:54.189003 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9879,"thread_start_us":11726,"threads_started":1}
I20260810 06:37:54.189719 8382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.174828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:44167 (local address 127.0.0.1:45570)
0810 06:37:54.185774 (+ 10946us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:54.185791 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:54.185876 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:54.186869 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:54.186874 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:54.186897 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:54.187712 (+ 815us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:37:54.187719 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:54.187749 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:37:54.187825 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:54.189027 (+ 1202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:54.189032 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:54.189037 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:37:54.189191 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 06:37:54.189247 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10831,"thread_start_us":10679,"threads_started":1}
I20260810 06:37:54.189935 8385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.176006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38666 (local address 127.31.59.195:45175)
0810 06:37:54.183688 (+ 7682us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:54.183708 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:54.186545 (+ 2837us) server_negotiation.cc:407] Connection header received
0810 06:37:54.186617 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:54.186623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:54.186801 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:54.186927 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:54.188510 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:54.188520 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:54.188533 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:54.188555 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:54.188619 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:54.188648 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:54.188823 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:54.189439 (+ 616us) server_negotiation.cc:299] Negotiation successful
0810 06:37:54.189551 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7477,"thread_start_us":7310,"threads_started":1}
I20260810 06:37:54.189647 8383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.175078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45570 (local address 127.31.59.193:44167)
0810 06:37:54.186355 (+ 11277us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:54.186364 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:54.186390 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:37:54.186453 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:54.186461 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:54.186681 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:54.187318 (+ 637us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:54.188742 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:54.188750 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:54.188762 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:54.188780 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:54.188835 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:54.188856 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:54.189024 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:54.189170 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 06:37:54.189267 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11108,"thread_start_us":10979,"threads_started":1}
I20260810 06:37:54.190414 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.175411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:45175 (local address 127.0.0.1:38666)
0810 06:37:54.186324 (+ 10913us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:54.186340 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:54.186439 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:54.187430 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:54.187435 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:54.187469 (+ 34us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:54.188248 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:37:54.188252 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:54.188268 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:37:54.188309 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:54.188887 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:54.188892 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:54.188897 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:37:54.189073 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 06:37:54.189119 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10775,"thread_start_us":8876,"threads_started":1}
I20260810 06:37:54.190577 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.174610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40300 (local address 127.31.59.194:38027)
0810 06:37:54.185502 (+ 10892us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:54.185513 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:54.185549 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:37:54.185627 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:54.185637 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:54.185879 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:54.186055 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:54.188176 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:54.188188 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:54.188206 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:54.188226 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:54.188293 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:54.188324 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:54.188496 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:54.190087 (+ 1591us) server_negotiation.cc:299] Negotiation successful
0810 06:37:54.190197 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10712,"thread_start_us":10579,"threads_started":1}
I20260810 06:37:54.193854 7772 tablet_service.cc:1467] Processing CreateTablet for tablet fd3346d41b4741eb81b442fd72f54152 (DEFAULT_TABLE table=test-table [id=e6c960ba94e345a4a8b250bab9ba4cbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:54.193935 8248 tablet_service.cc:1467] Processing CreateTablet for tablet fd3346d41b4741eb81b442fd72f54152 (DEFAULT_TABLE table=test-table [id=e6c960ba94e345a4a8b250bab9ba4cbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:54.193935 8026 tablet_service.cc:1467] Processing CreateTablet for tablet fd3346d41b4741eb81b442fd72f54152 (DEFAULT_TABLE table=test-table [id=e6c960ba94e345a4a8b250bab9ba4cbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:54.195173 7772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd3346d41b4741eb81b442fd72f54152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:54.195300 8248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd3346d41b4741eb81b442fd72f54152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:54.195300 8026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd3346d41b4741eb81b442fd72f54152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:54.207612 8389 tablet_bootstrap.cc:492] T fd3346d41b4741eb81b442fd72f54152 P b8b5766828b446ce85c2b4152478c32c: Bootstrap starting.
I20260810 06:37:54.209352 8391 tablet_bootstrap.cc:492] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a: Bootstrap starting.
I20260810 06:37:54.210178 8392 tablet_bootstrap.cc:492] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6: Bootstrap starting.
I20260810 06:37:54.211589 8389 tablet_bootstrap.cc:654] T fd3346d41b4741eb81b442fd72f54152 P b8b5766828b446ce85c2b4152478c32c: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:54.212963 8389 log.cc:824] T fd3346d41b4741eb81b442fd72f54152 P b8b5766828b446ce85c2b4152478c32c: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:54.213940 8392 tablet_bootstrap.cc:654] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:54.215250 8392 log.cc:824] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:54.216217 8389 tablet_bootstrap.cc:492] T fd3346d41b4741eb81b442fd72f54152 P b8b5766828b446ce85c2b4152478c32c: No bootstrap required, opened a new log
I20260810 06:37:54.216504 8389 ts_tablet_manager.cc:1397] T fd3346d41b4741eb81b442fd72f54152 P b8b5766828b446ce85c2b4152478c32c: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 06:37:54.216584 8391 tablet_bootstrap.cc:654] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:54.217834 8391 log.cc:824] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:54.218111 8392 tablet_bootstrap.cc:492] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6: No bootstrap required, opened a new log
I20260810 06:37:54.218407 8392 ts_tablet_manager.cc:1397] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260810 06:37:54.220594 8391 tablet_bootstrap.cc:492] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a: No bootstrap required, opened a new log
I20260810 06:37:54.220842 8391 ts_tablet_manager.cc:1397] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.002s
I20260810 06:37:54.225989 8392 raft_consensus.cc:348] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c84b32290b1403fac2a3c2ef153374a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38027 } } peers { permanent_uuid: "b8b5766828b446ce85c2b4152478c32c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44167 } } peers { permanent_uuid: "8b9c6df9b1de4b5eb83f21c1afff85d6" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45175 } }
I20260810 06:37:54.226369 8392 raft_consensus.cc:374] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:54.226509 8392 raft_consensus.cc:729] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b9c6df9b1de4b5eb83f21c1afff85d6, State: Initialized, Role: FOLLOWER
I20260810 06:37:54.226313 8389 raft_consensus.cc:348] T fd3346d41b4741eb81b442fd72f54152 P b8b5766828b446ce85c2b4152478c32c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c84b32290b1403fac2a3c2ef153374a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38027 } } peers { permanent_uuid: "b8b5766828b446ce85c2b4152478c32c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44167 } } peers { permanent_uuid: "8b9c6df9b1de4b5eb83f21c1afff85d6" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45175 } }
I20260810 06:37:54.226761 8389 raft_consensus.cc:374] T fd3346d41b4741eb81b442fd72f54152 P b8b5766828b446ce85c2b4152478c32c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:54.226881 8389 raft_consensus.cc:729] T fd3346d41b4741eb81b442fd72f54152 P b8b5766828b446ce85c2b4152478c32c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8b5766828b446ce85c2b4152478c32c, State: Initialized, Role: FOLLOWER
I20260810 06:37:54.227154 8392 consensus_queue.cc:260] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c84b32290b1403fac2a3c2ef153374a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38027 } } peers { permanent_uuid: "b8b5766828b446ce85c2b4152478c32c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44167 } } peers { permanent_uuid: "8b9c6df9b1de4b5eb83f21c1afff85d6" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45175 } }
I20260810 06:37:54.227444 8389 consensus_queue.cc:260] T fd3346d41b4741eb81b442fd72f54152 P b8b5766828b446ce85c2b4152478c32c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c84b32290b1403fac2a3c2ef153374a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38027 } } peers { permanent_uuid: "b8b5766828b446ce85c2b4152478c32c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44167 } } peers { permanent_uuid: "8b9c6df9b1de4b5eb83f21c1afff85d6" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45175 } }
I20260810 06:37:54.230858 8391 raft_consensus.cc:348] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c84b32290b1403fac2a3c2ef153374a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38027 } } peers { permanent_uuid: "b8b5766828b446ce85c2b4152478c32c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44167 } } peers { permanent_uuid: "8b9c6df9b1de4b5eb83f21c1afff85d6" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45175 } }
I20260810 06:37:54.231338 8391 raft_consensus.cc:374] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:54.231472 8391 raft_consensus.cc:729] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c84b32290b1403fac2a3c2ef153374a, State: Initialized, Role: FOLLOWER
I20260810 06:37:54.232178 8391 consensus_queue.cc:260] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c84b32290b1403fac2a3c2ef153374a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38027 } } peers { permanent_uuid: "b8b5766828b446ce85c2b4152478c32c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44167 } } peers { permanent_uuid: "8b9c6df9b1de4b5eb83f21c1afff85d6" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45175 } }
I20260810 06:37:54.235406 7898 heartbeater.cc:503] Master 127.31.59.254:39821 was elected leader, sending a full tablet report...
I20260810 06:37:54.235625 8389 ts_tablet_manager.cc:1428] T fd3346d41b4741eb81b442fd72f54152 P b8b5766828b446ce85c2b4152478c32c: Time spent starting tablet: real 0.019s user 0.017s sys 0.004s
I20260810 06:37:54.237520 8145 heartbeater.cc:503] Master 127.31.59.254:39821 was elected leader, sending a full tablet report...
I20260810 06:37:54.238418 8391 ts_tablet_manager.cc:1428] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a: Time spent starting tablet: real 0.017s user 0.015s sys 0.003s
I20260810 06:37:54.239223 8363 heartbeater.cc:503] Master 127.31.59.254:39821 was elected leader, sending a full tablet report...
I20260810 06:37:54.240891 8392 ts_tablet_manager.cc:1428] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6: Time spent starting tablet: real 0.022s user 0.019s sys 0.000s
I20260810 06:37:54.267438 8401 raft_consensus.cc:482] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:37:54.267676 8401 raft_consensus.cc:504] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c84b32290b1403fac2a3c2ef153374a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38027 } } peers { permanent_uuid: "b8b5766828b446ce85c2b4152478c32c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44167 } } peers { permanent_uuid: "8b9c6df9b1de4b5eb83f21c1afff85d6" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45175 } }
I20260810 06:37:54.269274 8401 leader_election.cc:288] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8b5766828b446ce85c2b4152478c32c (127.31.59.193:44167), 8b9c6df9b1de4b5eb83f21c1afff85d6 (127.31.59.195:45175)
I20260810 06:37:54.272426 8385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.269910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:60493 (local address 127.31.59.195:45175)
0810 06:37:54.270107 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:54.270114 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:54.270603 (+ 489us) server_negotiation.cc:407] Connection header received
0810 06:37:54.270759 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:54.270764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:54.270810 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:54.270888 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:54.271860 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:54.271868 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:54.271872 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:54.271886 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:54.271916 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:54.271927 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:54.272040 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:54.272122 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 06:37:54.272179 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:37:54.272527 8383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.269705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:58259 (local address 127.31.59.193:44167)
0810 06:37:54.270057 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:54.270062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:54.270851 (+ 789us) server_negotiation.cc:407] Connection header received
0810 06:37:54.270979 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:54.270984 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:54.271028 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:54.271105 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:54.271956 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:54.271962 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:54.271966 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:54.271984 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:54.272013 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:54.272024 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:54.272118 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:54.272204 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 06:37:54.272263 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236}
I20260810 06:37:54.272668 8402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.269543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:45175 (local address 127.31.59.194:60493)
0810 06:37:54.270490 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:54.270513 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:54.270645 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:54.270940 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:54.270945 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:54.270969 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:54.271641 (+ 672us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:37:54.271644 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:54.271662 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:37:54.271712 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:54.272018 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:54.272024 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:54.272029 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:37:54.272127 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 06:37:54.272176 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":586,"threads_started":1}
I20260810 06:37:54.272751 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.269654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:44167 (local address 127.31.59.194:58259)
0810 06:37:54.270754 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:54.270768 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:54.270871 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:54.271156 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:54.271161 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:54.271183 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:54.271782 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:37:54.271784 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:54.271796 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:37:54.271830 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:54.272107 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:54.272111 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:54.272115 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:37:54.272249 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 06:37:54.272295 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1007,"thread_start_us":811,"threads_started":1}
I20260810 06:37:54.273676 8281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd3346d41b4741eb81b442fd72f54152" candidate_uuid: "2c84b32290b1403fac2a3c2ef153374a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b9c6df9b1de4b5eb83f21c1afff85d6" is_pre_election: true
I20260810 06:37:54.273677 7808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd3346d41b4741eb81b442fd72f54152" candidate_uuid: "2c84b32290b1403fac2a3c2ef153374a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8b5766828b446ce85c2b4152478c32c" is_pre_election: true
I20260810 06:37:54.274210 8281 raft_consensus.cc:2454] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c84b32290b1403fac2a3c2ef153374a in term 0.
I20260810 06:37:54.274210 7808 raft_consensus.cc:2454] T fd3346d41b4741eb81b442fd72f54152 P b8b5766828b446ce85c2b4152478c32c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c84b32290b1403fac2a3c2ef153374a in term 0.
I20260810 06:37:54.274896 7937 leader_election.cc:302] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c84b32290b1403fac2a3c2ef153374a, b8b5766828b446ce85c2b4152478c32c; no voters:
I20260810 06:37:54.275236 8401 raft_consensus.cc:2781] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:37:54.275378 8401 raft_consensus.cc:482] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:37:54.275480 8401 raft_consensus.cc:3037] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:54.277580 8401 raft_consensus.cc:504] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c84b32290b1403fac2a3c2ef153374a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38027 } } peers { permanent_uuid: "b8b5766828b446ce85c2b4152478c32c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44167 } } peers { permanent_uuid: "8b9c6df9b1de4b5eb83f21c1afff85d6" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45175 } }
I20260810 06:37:54.278118 8401 leader_election.cc:288] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [CANDIDATE]: Term 1 election: Requested vote from peers b8b5766828b446ce85c2b4152478c32c (127.31.59.193:44167), 8b9c6df9b1de4b5eb83f21c1afff85d6 (127.31.59.195:45175)
I20260810 06:37:54.278585 7808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd3346d41b4741eb81b442fd72f54152" candidate_uuid: "2c84b32290b1403fac2a3c2ef153374a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8b5766828b446ce85c2b4152478c32c"
I20260810 06:37:54.278656 8281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd3346d41b4741eb81b442fd72f54152" candidate_uuid: "2c84b32290b1403fac2a3c2ef153374a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b9c6df9b1de4b5eb83f21c1afff85d6"
I20260810 06:37:54.278810 7808 raft_consensus.cc:3037] T fd3346d41b4741eb81b442fd72f54152 P b8b5766828b446ce85c2b4152478c32c [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:54.278822 8281 raft_consensus.cc:3037] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:54.281874 7808 raft_consensus.cc:2454] T fd3346d41b4741eb81b442fd72f54152 P b8b5766828b446ce85c2b4152478c32c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c84b32290b1403fac2a3c2ef153374a in term 1.
I20260810 06:37:54.281874 8281 raft_consensus.cc:2454] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c84b32290b1403fac2a3c2ef153374a in term 1.
I20260810 06:37:54.282240 7932 leader_election.cc:302] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c84b32290b1403fac2a3c2ef153374a, 8b9c6df9b1de4b5eb83f21c1afff85d6; no voters:
I20260810 06:37:54.282596 8401 raft_consensus.cc:2781] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:54.283102 8401 raft_consensus.cc:686] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [term 1 LEADER]: Becoming Leader. State: Replica: 2c84b32290b1403fac2a3c2ef153374a, State: Running, Role: LEADER
I20260810 06:37:54.283643 8401 consensus_queue.cc:237] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c84b32290b1403fac2a3c2ef153374a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38027 } } peers { permanent_uuid: "b8b5766828b446ce85c2b4152478c32c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44167 } } peers { permanent_uuid: "8b9c6df9b1de4b5eb83f21c1afff85d6" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45175 } }
I20260810 06:37:54.288061 7568 catalog_manager.cc:5495] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c84b32290b1403fac2a3c2ef153374a (127.31.59.194). New cstate: current_term: 1 leader_uuid: "2c84b32290b1403fac2a3c2ef153374a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c84b32290b1403fac2a3c2ef153374a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8b5766828b446ce85c2b4152478c32c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b9c6df9b1de4b5eb83f21c1afff85d6" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45175 } health_report { overall_health: UNKNOWN } } }
I20260810 06:37:54.314941 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.312974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40302 (local address 127.31.59.194:38027)
0810 06:37:54.313124 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:54.313128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:54.313231 (+ 103us) server_negotiation.cc:407] Connection header received
0810 06:37:54.313394 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:54.313397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:54.313469 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:54.313555 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:54.314390 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:54.314394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:54.314397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:54.314411 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:54.314440 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:54.314450 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:54.314550 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:54.314715 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 06:37:54.314761 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:37:54.322973 8281 raft_consensus.cc:1261] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6 [term 1 FOLLOWER]: Refusing update from remote peer 2c84b32290b1403fac2a3c2ef153374a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:54.322973 7808 raft_consensus.cc:1261] T fd3346d41b4741eb81b442fd72f54152 P b8b5766828b446ce85c2b4152478c32c [term 1 FOLLOWER]: Refusing update from remote peer 2c84b32290b1403fac2a3c2ef153374a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:54.323952 8401 consensus_queue.cc:1035] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b9c6df9b1de4b5eb83f21c1afff85d6" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:54.324257 8405 consensus_queue.cc:1035] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8b5766828b446ce85c2b4152478c32c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:54.337277 8414 mvcc.cc:205] Tried to move back new op lower bound from 7316864509216223232 to 7316864509069778944. Current Snapshot: MvccSnapshot[applied={T|T < 7316864509216223232}]
I20260810 06:37:54.346535 7569 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:37:54.346762 7569 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:37:54.350100 7569 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d7a46e82d0614ef783157ede220e9540: Sending DeleteTablet for 3 replicas of tablet fd3346d41b4741eb81b442fd72f54152
I20260810 06:37:54.351110 8026 tablet_service.cc:1514] Processing DeleteTablet for tablet fd3346d41b4741eb81b442fd72f54152 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:54 UTC) from {username='slave'} at 127.0.0.1:40300
I20260810 06:37:54.351164 8248 tablet_service.cc:1514] Processing DeleteTablet for tablet fd3346d41b4741eb81b442fd72f54152 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:54 UTC) from {username='slave'} at 127.0.0.1:38666
I20260810 06:37:54.351203 7772 tablet_service.cc:1514] Processing DeleteTablet for tablet fd3346d41b4741eb81b442fd72f54152 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:54 UTC) from {username='slave'} at 127.0.0.1:45570
I20260810 06:37:54.353521 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 7664
I20260810 06:37:54.362994 8383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.354590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45580 (local address 127.31.59.193:44167)
0810 06:37:54.354771 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:54.354777 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:54.359746 (+ 4969us) server_negotiation.cc:407] Connection header received
0810 06:37:54.359934 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:54.359940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:54.360063 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:54.360154 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:54.361920 (+ 1766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:54.361926 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:54.361932 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:54.361954 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:54.361995 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:54.362018 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:54.362116 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:54.362586 (+ 470us) server_negotiation.cc:299] Negotiation successful
0810 06:37:54.362656 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:37:54.364260 7893 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:54.380247 8422 tablet_replica.cc:330] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6: stopping tablet replica
I20260810 06:37:54.380518 8423 tablet_replica.cc:330] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a: stopping tablet replica
I20260810 06:37:54.380744 8422 raft_consensus.cc:2229] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:37:54.380965 8423 raft_consensus.cc:2229] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:37:54.381314 8422 raft_consensus.cc:2258] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:54.381672 8423 raft_consensus.cc:2258] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:54.383586 8423 ts_tablet_manager.cc:1905] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:54.383579 8422 ts_tablet_manager.cc:1905] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:54.390269 8422 ts_tablet_manager.cc:1918] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:37:54.390529 8422 log.cc:1196] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/fd3346d41b4741eb81b442fd72f54152
I20260810 06:37:54.390774 8423 ts_tablet_manager.cc:1918] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:37:54.390970 8423 log.cc:1196] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/fd3346d41b4741eb81b442fd72f54152
I20260810 06:37:54.391227 8422 ts_tablet_manager.cc:1939] T fd3346d41b4741eb81b442fd72f54152 P 8b9c6df9b1de4b5eb83f21c1afff85d6: Deleting consensus metadata
I20260810 06:37:54.391522 8423 ts_tablet_manager.cc:1939] T fd3346d41b4741eb81b442fd72f54152 P 2c84b32290b1403fac2a3c2ef153374a: Deleting consensus metadata
I20260810 06:37:54.393092 7548 catalog_manager.cc:4841] TS 2c84b32290b1403fac2a3c2ef153374a (127.31.59.194:38027): tablet fd3346d41b4741eb81b442fd72f54152 (table test-table [id=e6c960ba94e345a4a8b250bab9ba4cbf]) successfully deleted
I20260810 06:37:54.393536 7547 catalog_manager.cc:4841] TS 8b9c6df9b1de4b5eb83f21c1afff85d6 (127.31.59.195:45175): tablet fd3346d41b4741eb81b442fd72f54152 (table test-table [id=e6c960ba94e345a4a8b250bab9ba4cbf]) successfully deleted
I20260810 06:37:54.519197 8424 tablet_replica.cc:330] T fd3346d41b4741eb81b442fd72f54152 P b8b5766828b446ce85c2b4152478c32c: stopping tablet replica
I20260810 06:37:54.519933 8424 raft_consensus.cc:2229] T fd3346d41b4741eb81b442fd72f54152 P b8b5766828b446ce85c2b4152478c32c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:37:54.520124 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 7664
W20260810 06:37:54.550424 7551 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.59.193:44167 (error 108)
I20260810 06:37:54.550694 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 7904
I20260810 06:37:54.552748 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.552169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:44167 (local address 127.0.0.1:45590)
0810 06:37:54.552364 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:54.552576 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:44167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260810 06:37:54.553004 7551 catalog_manager.cc:4584] TS b8b5766828b446ce85c2b4152478c32c (127.31.59.193:44167): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fd3346d41b4741eb81b442fd72f54152: Network error: Client connection negotiation failed: client connection to 127.31.59.193:44167: connect: Connection refused (error 111)
I20260810 06:37:54.553995 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.551988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40314 (local address 127.31.59.194:38027)
0810 06:37:54.552195 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:54.552200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:54.552221 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:37:54.552309 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:54.552314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:54.552377 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:54.552471 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:54.553368 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:54.553374 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:54.553380 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:54.553398 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:54.553432 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:54.553444 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:54.553570 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:54.553676 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 06:37:54.553737 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:37:54.554714 8130 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:54.606858 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.606263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:44167 (local address 127.0.0.1:45596)
0810 06:37:54.606539 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:54.606665 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:44167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260810 06:37:54.607946 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.607557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:44167 (local address 127.0.0.1:45612)
0810 06:37:54.607716 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:54.607801 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:44167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 06:37:54.661109 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 7904
I20260810 06:37:54.668409 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.667936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:44167 (local address 127.0.0.1:45626)
0810 06:37:54.668142 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:54.668232 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:44167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 06:37:54.669543 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.669082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:44167 (local address 127.0.0.1:45634)
0810 06:37:54.669252 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:54.669351 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:44167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 06:37:54.686354 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 8151
I20260810 06:37:54.689013 8385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.687055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38682 (local address 127.31.59.195:45175)
0810 06:37:54.687241 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:54.687245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:54.687264 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:37:54.687369 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:54.687372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:54.687446 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:54.687553 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:54.688395 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:54.688401 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:54.688406 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:54.688423 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:54.688457 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:54.688469 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:54.688582 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:54.688690 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 06:37:54.688752 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:37:54.689642 8352 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:54.750165 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.749646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:44167 (local address 127.0.0.1:45638)
0810 06:37:54.749885 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:54.750008 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:44167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 06:37:54.751241 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:54.750834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:44167 (local address 127.0.0.1:45652)
0810 06:37:54.751057 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:54.751128 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:44167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260810 06:37:54.779240 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 8151
I20260810 06:37:54.801101 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 7528
I20260810 06:37:54.802233 7617 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:54.907290 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 7528
2026-08-10T06:37:54Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2682 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 06:37:54.959064 31983 mini_oidc.cc:152] Starting JWKS server
I20260810 06:37:54.962147 31983 webserver.cc:460] Webserver started at https://127.0.0.1:35583/ using document root <none> and password file <none>
I20260810 06:37:54.962581 31983 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 06:37:54.963289 31983 webserver.cc:460] Webserver started at http://127.0.0.1:46843/ using document root <none> and password file <none>
2026-08-10T06:37:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:37:54Z Disabled control of system clock
I20260810 06:37:54.991628 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40573
--webserver_interface=127.31.59.254
--webserver_port=0
--builtin_ntp_servers=127.31.59.212:39101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:40573
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35583/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/testchainca.cert with env {}
W20260810 06:37:55.305557 8446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:55.305832 8446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:55.305968 8446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:55.312822 8446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:37:55.312932 8446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:55.313006 8446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:55.313086 8446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:37:55.313144 8446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:37:55.313232 8446 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:37:55.313285 8446 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35583/jwks.json
I20260810 06:37:55.322841 8446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:39101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:40573
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35583/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:55.324100 8446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:55.325400 8446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:55.336444 8459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:55.336508 8458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:55.336555 8463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:55.337597 8446 server_base.cc:1034] running on GCE node
I20260810 06:37:55.338376 8446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:55.339715 8446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:55.340910 8446 hybrid_clock.cc:648] HybridClock initialized: now 1786343875340867 us; error 33 us; skew 500 ppm
I20260810 06:37:55.341244 8446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:37:55.343361 8446 webserver.cc:460] Webserver started at http://127.31.59.254:40355/ using document root <none> and password file <none>
I20260810 06:37:55.344023 8446 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:55.344125 8446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:55.344421 8446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:55.347294 8446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "151670643bed40d7b56e4f7c38ab14d4"
format_stamp: "Formatted at 2026-08-10 06:37:55 on dist-test-slave-4tdj"
server_key: "e2d40df25f497aba42e40f9cbf2f706c"
server_key_iv: "c5fc1540d1e3c04fd596f30cfef9cbd1"
server_key_version: "encryptionkey@0"
I20260810 06:37:55.348055 8446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "151670643bed40d7b56e4f7c38ab14d4"
format_stamp: "Formatted at 2026-08-10 06:37:55 on dist-test-slave-4tdj"
server_key: "e2d40df25f497aba42e40f9cbf2f706c"
server_key_iv: "c5fc1540d1e3c04fd596f30cfef9cbd1"
server_key_version: "encryptionkey@0"
I20260810 06:37:55.352856 8446 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260810 06:37:55.356164 8473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:55.357537 8446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:37:55.357689 8446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "151670643bed40d7b56e4f7c38ab14d4"
format_stamp: "Formatted at 2026-08-10 06:37:55 on dist-test-slave-4tdj"
server_key: "e2d40df25f497aba42e40f9cbf2f706c"
server_key_iv: "c5fc1540d1e3c04fd596f30cfef9cbd1"
server_key_version: "encryptionkey@0"
I20260810 06:37:55.357884 8446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:55.381989 8446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:55.382819 8446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:55.383108 8446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:55.400069 8446 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:40573
I20260810 06:37:55.400089 8551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:40573 every 8 connection(s)
I20260810 06:37:55.401336 8446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:37:55.404933 8552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:55.407496 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 8446
I20260810 06:37:55.407670 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:37:55.408288 31983 external_mini_cluster.cc:1428] Setting key c8fe27d87563509068ce25b695055a46
I20260810 06:37:55.417646 8552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4: Bootstrap starting.
I20260810 06:37:55.421276 8552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:55.422428 8552 log.cc:824] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:55.423028 8556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:55.410119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54696 (local address 127.31.59.254:40573)
0810 06:37:55.411160 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:55.411174 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:55.411224 (+ 50us) server_negotiation.cc:407] Connection header received
0810 06:37:55.413280 (+ 2056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:55.413307 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:55.413889 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:55.414336 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:55.414856 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:55.416793 (+ 1937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:55.418165 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:55.419155 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:55.419359 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:55.420367 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:55.420378 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:55.420396 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:55.421051 (+ 655us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:55.421171 (+ 120us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:55.421197 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:55.421342 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:55.421691 (+ 349us) server_negotiation.cc:299] Negotiation successful
0810 06:37:55.421989 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":332,"threads_started":1}
I20260810 06:37:55.425055 8552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4: No bootstrap required, opened a new log
I20260810 06:37:55.426426 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:37:55.432683 8552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151670643bed40d7b56e4f7c38ab14d4" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40573 } }
I20260810 06:37:55.433018 8552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:55.433109 8552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 151670643bed40d7b56e4f7c38ab14d4, State: Initialized, Role: FOLLOWER
I20260810 06:37:55.433619 8552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151670643bed40d7b56e4f7c38ab14d4" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40573 } }
I20260810 06:37:55.433849 8552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:37:55.433987 8552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:37:55.434130 8552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:55.436479 8552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151670643bed40d7b56e4f7c38ab14d4" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40573 } }
I20260810 06:37:55.436954 8552 leader_election.cc:302] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 151670643bed40d7b56e4f7c38ab14d4; no voters:
I20260810 06:37:55.437462 8552 leader_election.cc:288] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:37:55.437613 8560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:55.438246 8560 raft_consensus.cc:686] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4 [term 1 LEADER]: Becoming Leader. State: Replica: 151670643bed40d7b56e4f7c38ab14d4, State: Running, Role: LEADER
I20260810 06:37:55.438791 8560 consensus_queue.cc:237] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151670643bed40d7b56e4f7c38ab14d4" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40573 } }
I20260810 06:37:55.439359 8552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:37:55.441538 8561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "151670643bed40d7b56e4f7c38ab14d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151670643bed40d7b56e4f7c38ab14d4" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40573 } } }
I20260810 06:37:55.441514 8562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 151670643bed40d7b56e4f7c38ab14d4. Latest consensus state: current_term: 1 leader_uuid: "151670643bed40d7b56e4f7c38ab14d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151670643bed40d7b56e4f7c38ab14d4" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40573 } } }
I20260810 06:37:55.441998 8561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:55.442027 8562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:55.443743 8567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:37:55.447779 8567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:37:55.456424 8567 catalog_manager.cc:1347] Generated new cluster ID: 199d2506a16b420ab1c5ab76184e554f
I20260810 06:37:55.456527 8567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:37:55.465171 8567 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:37:55.466413 8567 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:37:55.490729 8567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 151670643bed40d7b56e4f7c38ab14d4: Generated new TSK 0
I20260810 06:37:55.491812 8567 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 06:37:55.538098 8596 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 06:37:55.538324 8596 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 06:37:55.539727 8556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:55.517033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54706 (local address 127.31.59.254:40573)
0810 06:37:55.517251 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:55.517257 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:55.517604 (+ 347us) server_negotiation.cc:407] Connection header received
0810 06:37:55.517934 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:55.517941 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:55.518049 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:55.518227 (+ 178us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:37:55.518785 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:55.519898 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:55.521293 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:55.522018 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:55.522157 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:55.522790 (+ 633us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:37:55.538927 (+ 16137us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35583/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 06:37:55.539388 (+ 461us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54706: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35583/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":75}
W20260810 06:37:55.539866 8556 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54706: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35583/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 06:37:55.541356 8595 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:37:55.516877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40573 (local address 127.0.0.1:54706)
0810 06:37:55.517496 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:55.517516 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:55.517655 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:55.518254 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:55.518257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:55.518267 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:37:55.518638 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:55.518648 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:55.520052 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:55.520056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:55.521150 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:55.521161 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:55.522214 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:55.522220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:55.522410 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:55.522469 (+ 59us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:37:55.539248 (+ 16779us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35583/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 06:37:55.539572 (+ 324us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.59.254:40573: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35583/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":307,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.31.59.254:40573: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35583/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 06:37:55.542994 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 8446
I20260810 06:37:55.544188 8545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:55.636883 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 8446
2026-08-10T06:37:55Z chronyd exiting
I20260810 06:37:55.691782 31983 test_util.cc:175] -----------------------------------------------
I20260810 06:37:55.691908 31983 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343831713822-31983-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (737 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 06:37:55.695670 31983 mini_oidc.cc:152] Starting JWKS server
I20260810 06:37:55.697471 31983 webserver.cc:460] Webserver started at https://127.0.0.1:45441/ using document root <none> and password file <none>
I20260810 06:37:55.697695 31983 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 06:37:55.698391 31983 webserver.cc:460] Webserver started at http://127.0.0.1:35559/ using document root <none> and password file <none>
2026-08-10T06:37:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:37:55Z Disabled control of system clock
I20260810 06:37:55.726955 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:34001
--webserver_interface=127.31.59.254
--webserver_port=0
--builtin_ntp_servers=127.31.59.212:35697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:34001
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45441/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/testchainca.cert with env {}
W20260810 06:37:56.042457 8609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:56.042730 8609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:56.042857 8609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:56.049822 8609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:37:56.049921 8609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:56.049978 8609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:56.050050 8609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:37:56.050096 8609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:37:56.050179 8609 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:37:56.050233 8609 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45441/jwks.json
I20260810 06:37:56.060081 8609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:35697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:34001
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:34001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45441/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:56.061275 8609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:56.062593 8609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:56.073665 8620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:56.073628 8621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:56.073738 8624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:56.074611 8609 server_base.cc:1034] running on GCE node
I20260810 06:37:56.075346 8609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:56.076587 8609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:56.077783 8609 hybrid_clock.cc:648] HybridClock initialized: now 1786343876077743 us; error 32 us; skew 500 ppm
I20260810 06:37:56.078110 8609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:37:56.080390 8609 webserver.cc:460] Webserver started at http://127.31.59.254:42985/ using document root <none> and password file <none>
I20260810 06:37:56.081035 8609 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:56.081141 8609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:56.081445 8609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:56.084211 8609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "1735ec61966d44b58af8deaa331f5fd9"
format_stamp: "Formatted at 2026-08-10 06:37:56 on dist-test-slave-4tdj"
server_key: "1f725651957279a1df6eecf227bce85d"
server_key_iv: "0a142e1e2b2fa365c8cd33411c0e183d"
server_key_version: "encryptionkey@0"
I20260810 06:37:56.084991 8609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "1735ec61966d44b58af8deaa331f5fd9"
format_stamp: "Formatted at 2026-08-10 06:37:56 on dist-test-slave-4tdj"
server_key: "1f725651957279a1df6eecf227bce85d"
server_key_iv: "0a142e1e2b2fa365c8cd33411c0e183d"
server_key_version: "encryptionkey@0"
I20260810 06:37:56.089881 8609 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 06:37:56.093206 8632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:56.094501 8609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:37:56.094664 8609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "1735ec61966d44b58af8deaa331f5fd9"
format_stamp: "Formatted at 2026-08-10 06:37:56 on dist-test-slave-4tdj"
server_key: "1f725651957279a1df6eecf227bce85d"
server_key_iv: "0a142e1e2b2fa365c8cd33411c0e183d"
server_key_version: "encryptionkey@0"
I20260810 06:37:56.094854 8609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:56.114859 8609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:56.115643 8609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:56.115921 8609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:56.132966 8609 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:34001
I20260810 06:37:56.132998 8719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:34001 every 8 connection(s)
I20260810 06:37:56.134248 8609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:37:56.138077 8720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:56.143452 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 8609
I20260810 06:37:56.143646 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:37:56.144268 31983 external_mini_cluster.cc:1428] Setting key 35587c7bbf58538bf544c6d80d96c277
I20260810 06:37:56.148793 8720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9: Bootstrap starting.
I20260810 06:37:56.151784 8720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:56.152738 8720 log.cc:824] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:56.155495 8720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9: No bootstrap required, opened a new log
I20260810 06:37:56.156781 8723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:56.146125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39674 (local address 127.31.59.254:34001)
0810 06:37:56.147121 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:56.147131 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:56.147171 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:37:56.148770 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:56.148794 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:56.149232 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:56.149560 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:56.150107 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:56.151378 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:56.152688 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:56.153390 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:56.153560 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:56.154657 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:56.154665 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:56.154679 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:56.155173 (+ 494us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:56.155250 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:56.155270 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:56.155368 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:56.155587 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 06:37:56.155887 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":347,"threads_started":1}
I20260810 06:37:56.160327 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:37:56.162995 8720 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1735ec61966d44b58af8deaa331f5fd9" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34001 } }
I20260810 06:37:56.163357 8720 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:56.163468 8720 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1735ec61966d44b58af8deaa331f5fd9, State: Initialized, Role: FOLLOWER
I20260810 06:37:56.163970 8720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1735ec61966d44b58af8deaa331f5fd9" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34001 } }
I20260810 06:37:56.164173 8720 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:37:56.164280 8720 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:37:56.164407 8720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:56.166723 8720 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1735ec61966d44b58af8deaa331f5fd9" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34001 } }
I20260810 06:37:56.167135 8720 leader_election.cc:302] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1735ec61966d44b58af8deaa331f5fd9; no voters:
I20260810 06:37:56.167713 8720 leader_election.cc:288] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:37:56.167784 8727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:56.168823 8727 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9 [term 1 LEADER]: Becoming Leader. State: Replica: 1735ec61966d44b58af8deaa331f5fd9, State: Running, Role: LEADER
I20260810 06:37:56.169291 8727 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1735ec61966d44b58af8deaa331f5fd9" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34001 } }
I20260810 06:37:56.169652 8720 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:37:56.171913 8728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1735ec61966d44b58af8deaa331f5fd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1735ec61966d44b58af8deaa331f5fd9" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34001 } } }
I20260810 06:37:56.171922 8729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1735ec61966d44b58af8deaa331f5fd9. Latest consensus state: current_term: 1 leader_uuid: "1735ec61966d44b58af8deaa331f5fd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1735ec61966d44b58af8deaa331f5fd9" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34001 } } }
I20260810 06:37:56.172293 8728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:56.172287 8729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:56.173839 8740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:37:56.178789 8740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:37:56.187366 8740 catalog_manager.cc:1347] Generated new cluster ID: c2b9bab3b5274521b115ee11788272af
I20260810 06:37:56.187467 8740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:37:56.201164 8740 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:37:56.202584 8740 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:37:56.215220 8740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1735ec61966d44b58af8deaa331f5fd9: Generated new TSK 0
I20260810 06:37:56.216013 8740 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 06:37:56.252804 8766 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 06:37:56.252913 8766 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 06:37:56.254154 8723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:56.234454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39682 (local address 127.31.59.254:34001)
0810 06:37:56.234639 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:56.234645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:56.235038 (+ 393us) server_negotiation.cc:407] Connection header received
0810 06:37:56.235263 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:56.235268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:56.235369 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:56.235538 (+ 169us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:37:56.235931 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:56.237049 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:56.238451 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:56.239132 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:56.239264 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:56.239715 (+ 451us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:37:56.253503 (+ 13788us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45441/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 06:37:56.253872 (+ 369us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39682: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45441/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 06:37:56.254206 8765 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:37:56.234276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:34001 (local address 127.0.0.1:39682)
0810 06:37:56.234938 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:56.234954 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:56.235089 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:56.235533 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:56.235536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:56.235545 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:37:56.235776 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:56.235784 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:56.237200 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:56.237205 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:56.238259 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:56.238271 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:56.239300 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:56.239304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:56.239431 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:56.239463 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:37:56.253718 (+ 14255us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45441/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 06:37:56.253899 (+ 181us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.59.254:34001: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45441/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":358,"threads_started":1}
W20260810 06:37:56.254333 8723 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39682: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45441/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260810 06:37:56.256013 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 8609
I20260810 06:37:56.257014 8715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:56.348703 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 8609
2026-08-10T06:37:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (715 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 06:37:56.411432 31983 mini_oidc.cc:152] Starting JWKS server
I20260810 06:37:56.413249 31983 webserver.cc:460] Webserver started at https://127.0.0.1:46385/ using document root <none> and password file <none>
I20260810 06:37:56.413429 31983 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 06:37:56.414036 31983 webserver.cc:460] Webserver started at http://127.0.0.1:33181/ using document root <none> and password file <none>
2026-08-10T06:37:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:37:56Z Disabled control of system clock
I20260810 06:37:56.441947 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:45149
--webserver_interface=127.31.59.254
--webserver_port=0
--builtin_ntp_servers=127.31.59.212:40985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:45149
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46385/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 06:37:56.755505 8776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:56.755781 8776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:56.755909 8776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:56.762821 8776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:37:56.762914 8776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:56.762975 8776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:56.763049 8776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:37:56.763108 8776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:37:56.763198 8776 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:37:56.763242 8776 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46385/jwks.json
I20260810 06:37:56.772213 8776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:40985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:45149
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:45149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46385/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:56.773386 8776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:56.775168 8776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:56.786149 8786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:56.786207 8789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:56.786180 8787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:56.786952 8776 server_base.cc:1034] running on GCE node
I20260810 06:37:56.787695 8776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:56.788980 8776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:56.790177 8776 hybrid_clock.cc:648] HybridClock initialized: now 1786343876790135 us; error 35 us; skew 500 ppm
I20260810 06:37:56.790529 8776 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:37:56.792650 8776 webserver.cc:460] Webserver started at http://127.31.59.254:41243/ using document root <none> and password file <none>
I20260810 06:37:56.793289 8776 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:56.793382 8776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:56.793700 8776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:56.796527 8776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "bbb873280361482aad18a655093aff79"
format_stamp: "Formatted at 2026-08-10 06:37:56 on dist-test-slave-4tdj"
server_key: "8717332794cde7100591ade9d4f222cb"
server_key_iv: "329647a8db75a6465714550c9207ff10"
server_key_version: "encryptionkey@0"
I20260810 06:37:56.797297 8776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "bbb873280361482aad18a655093aff79"
format_stamp: "Formatted at 2026-08-10 06:37:56 on dist-test-slave-4tdj"
server_key: "8717332794cde7100591ade9d4f222cb"
server_key_iv: "329647a8db75a6465714550c9207ff10"
server_key_version: "encryptionkey@0"
I20260810 06:37:56.802189 8776 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 06:37:56.805573 8796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:56.806880 8776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:37:56.807037 8776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "bbb873280361482aad18a655093aff79"
format_stamp: "Formatted at 2026-08-10 06:37:56 on dist-test-slave-4tdj"
server_key: "8717332794cde7100591ade9d4f222cb"
server_key_iv: "329647a8db75a6465714550c9207ff10"
server_key_version: "encryptionkey@0"
I20260810 06:37:56.807227 8776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:56.859404 8776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:56.860265 8776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:56.860577 8776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:56.877902 8776 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:45149
I20260810 06:37:56.877971 8887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:45149 every 8 connection(s)
I20260810 06:37:56.879215 8776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:37:56.882992 8888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:56.888574 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 8776
I20260810 06:37:56.888749 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:37:56.889358 31983 external_mini_cluster.cc:1428] Setting key ad3d190dbee7cd3a2fbb87c3fed808e1
I20260810 06:37:56.894795 8888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79: Bootstrap starting.
I20260810 06:37:56.899001 8888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:56.900074 8888 log.cc:824] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:56.903021 8888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79: No bootstrap required, opened a new log
I20260810 06:37:56.904824 8891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:56.891401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45124 (local address 127.31.59.254:45149)
0810 06:37:56.892516 (+ 1115us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:56.892531 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:56.892585 (+ 54us) server_negotiation.cc:407] Connection header received
0810 06:37:56.894986 (+ 2401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:56.895013 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:56.895577 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:56.896004 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:56.896604 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:56.898649 (+ 2045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:56.900044 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:56.901057 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:56.901255 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:56.902492 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:56.902500 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:56.902518 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:56.903051 (+ 533us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:56.903138 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:56.903166 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:56.903286 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:56.903553 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 06:37:56.903851 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":304,"threads_started":1}
I20260810 06:37:56.908825 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:37:56.911170 8888 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb873280361482aad18a655093aff79" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 45149 } }
I20260810 06:37:56.911597 8888 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:56.911710 8888 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbb873280361482aad18a655093aff79, State: Initialized, Role: FOLLOWER
I20260810 06:37:56.912328 8888 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb873280361482aad18a655093aff79" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 45149 } }
I20260810 06:37:56.912554 8888 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:37:56.912660 8888 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:37:56.912782 8888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:56.914878 8888 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb873280361482aad18a655093aff79" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 45149 } }
I20260810 06:37:56.915298 8888 leader_election.cc:302] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbb873280361482aad18a655093aff79; no voters:
I20260810 06:37:56.915882 8888 leader_election.cc:288] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:37:56.916003 8894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:56.916931 8894 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79 [term 1 LEADER]: Becoming Leader. State: Replica: bbb873280361482aad18a655093aff79, State: Running, Role: LEADER
I20260810 06:37:56.917506 8894 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb873280361482aad18a655093aff79" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 45149 } }
I20260810 06:37:56.917999 8888 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:37:56.920517 8896 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbb873280361482aad18a655093aff79. Latest consensus state: current_term: 1 leader_uuid: "bbb873280361482aad18a655093aff79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb873280361482aad18a655093aff79" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 45149 } } }
I20260810 06:37:56.920903 8896 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:56.923203 8907 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:37:56.923034 8895 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbb873280361482aad18a655093aff79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb873280361482aad18a655093aff79" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 45149 } } }
I20260810 06:37:56.923388 8895 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79 [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:56.928416 8907 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:37:56.937490 8907 catalog_manager.cc:1347] Generated new cluster ID: d33773b86b1a45d3b924cb287d1769f7
I20260810 06:37:56.937608 8907 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:37:56.956240 8907 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:37:56.958112 8907 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:37:56.970443 8907 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbb873280361482aad18a655093aff79: Generated new TSK 0
I20260810 06:37:56.971469 8907 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:37:56.997252 8891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:56.989141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45134 (local address 127.31.59.254:45149)
0810 06:37:56.989328 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:56.989335 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:56.989747 (+ 412us) server_negotiation.cc:407] Connection header received
0810 06:37:56.989959 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:56.989965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:56.990068 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:56.990221 (+ 153us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:37:56.990695 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:56.991781 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:56.993064 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:56.993733 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:56.993863 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:56.994358 (+ 495us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:37:56.996631 (+ 2273us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46385/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 06:37:56.996936 (+ 305us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45134: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46385/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":58}
W20260810 06:37:56.997375 8891 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45134: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46385/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 06:37:56.998975 8933 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:37:56.989050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:45149 (local address 127.0.0.1:45134)
0810 06:37:56.989654 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:56.989670 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:56.989787 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:56.990223 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:56.990226 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:56.990235 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:37:56.990545 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:56.990557 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:56.991903 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:56.991907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:56.992920 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:56.992932 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:56.993909 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:56.993914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:56.994054 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:56.994086 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:37:56.996837 (+ 2751us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46385/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 06:37:56.998533 (+ 1696us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.59.254:45149: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46385/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":1423,"thread_start_us":340,"threads_started":1}
I20260810 06:37:57.000129 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 8776
I20260810 06:37:57.001302 8881 generic_service.cc:196] Checking for leaks (request via RPC)
E20260810 06:37:57.008857 8934 webserver.cc:533] Webserver: sslize failed:
E20260810 06:37:57.008950 8934 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 06:37:57.093108 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 8776
2026-08-10T06:37:57Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (742 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T06:37:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:37:57Z Disabled control of system clock
I20260810 06:37:57.181792 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:44609
--webserver_interface=127.31.59.254
--webserver_port=0
--builtin_ntp_servers=127.31.59.212:42981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:44609
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/testchainca.cert with env {}
W20260810 06:37:57.493861 8946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:57.494140 8946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:57.494234 8946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:57.501147 8946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:37:57.501241 8946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:57.501276 8946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:57.501329 8946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:37:57.501372 8946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:37:57.501431 8946 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:37:57.501468 8946 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 06:37:57.510852 8946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:42981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:44609
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:44609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:37:57.511981 8946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:57.513159 8946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:57.524175 8953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:57.524227 8954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:57.524263 8958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:57.525285 8946 server_base.cc:1034] running on GCE node
I20260810 06:37:57.525980 8946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:57.527365 8946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:57.528581 8946 hybrid_clock.cc:648] HybridClock initialized: now 1786343877528537 us; error 37 us; skew 500 ppm
I20260810 06:37:57.528882 8946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:37:57.531046 8946 webserver.cc:460] Webserver started at http://127.31.59.254:46765/ using document root <none> and password file <none>
I20260810 06:37:57.531666 8946 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:57.531752 8946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:57.532011 8946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:37:57.534771 8946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "7b63043ec77940afb4cc3076a804ae9d"
format_stamp: "Formatted at 2026-08-10 06:37:57 on dist-test-slave-4tdj"
server_key: "7560b5cc999ccacf04a7c85ef657ce59"
server_key_iv: "9cddd7ae6d0f25c29a420a8300489fa1"
server_key_version: "encryptionkey@0"
I20260810 06:37:57.535504 8946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "7b63043ec77940afb4cc3076a804ae9d"
format_stamp: "Formatted at 2026-08-10 06:37:57 on dist-test-slave-4tdj"
server_key: "7560b5cc999ccacf04a7c85ef657ce59"
server_key_iv: "9cddd7ae6d0f25c29a420a8300489fa1"
server_key_version: "encryptionkey@0"
I20260810 06:37:57.540448 8946 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:37:57.543740 8967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:57.545042 8946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:37:57.545197 8946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "7b63043ec77940afb4cc3076a804ae9d"
format_stamp: "Formatted at 2026-08-10 06:37:57 on dist-test-slave-4tdj"
server_key: "7560b5cc999ccacf04a7c85ef657ce59"
server_key_iv: "9cddd7ae6d0f25c29a420a8300489fa1"
server_key_version: "encryptionkey@0"
I20260810 06:37:57.545399 8946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:57.570919 8946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:57.571713 8946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:57.571972 8946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:57.589232 8946 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:44609
I20260810 06:37:57.589267 9062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:44609 every 8 connection(s)
I20260810 06:37:57.590528 8946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:37:57.594372 9063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:57.598613 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 8946
I20260810 06:37:57.598801 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:37:57.599398 31983 external_mini_cluster.cc:1428] Setting key 5f4a9fe6b3b6e0e52e8de274dc7de473
I20260810 06:37:57.605844 9063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d: Bootstrap starting.
I20260810 06:37:57.609561 9063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:57.610749 9063 log.cc:824] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:57.613234 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:57.601176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51092 (local address 127.31.59.254:44609)
0810 06:37:57.602061 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:57.602072 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:57.602117 (+ 45us) server_negotiation.cc:407] Connection header received
0810 06:37:57.604097 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:57.604118 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:57.604542 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:57.604854 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:57.605543 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:57.607070 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:57.608933 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:57.609587 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:57.609757 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:57.610958 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:57.610968 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:57.610985 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:37:57.611631 (+ 646us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:57.611726 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:37:57.611751 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:57.611882 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:57.612101 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 06:37:57.612373 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":303,"threads_started":1}
I20260810 06:37:57.613788 9063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d: No bootstrap required, opened a new log
I20260810 06:37:57.616683 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:37:57.622610 9063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b63043ec77940afb4cc3076a804ae9d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 44609 } }
I20260810 06:37:57.622944 9063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:57.623034 9063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b63043ec77940afb4cc3076a804ae9d, State: Initialized, Role: FOLLOWER
I20260810 06:37:57.623598 9063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b63043ec77940afb4cc3076a804ae9d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 44609 } }
I20260810 06:37:57.623816 9063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:37:57.623922 9063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:37:57.624073 9063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:57.626425 9063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b63043ec77940afb4cc3076a804ae9d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 44609 } }
I20260810 06:37:57.626947 9063 leader_election.cc:302] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b63043ec77940afb4cc3076a804ae9d; no voters:
I20260810 06:37:57.627499 9063 leader_election.cc:288] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:37:57.627662 9068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:57.628331 9068 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d [term 1 LEADER]: Becoming Leader. State: Replica: 7b63043ec77940afb4cc3076a804ae9d, State: Running, Role: LEADER
I20260810 06:37:57.628850 9068 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b63043ec77940afb4cc3076a804ae9d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 44609 } }
I20260810 06:37:57.629526 9063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:37:57.631867 9072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b63043ec77940afb4cc3076a804ae9d. Latest consensus state: current_term: 1 leader_uuid: "7b63043ec77940afb4cc3076a804ae9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b63043ec77940afb4cc3076a804ae9d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 44609 } } }
I20260810 06:37:57.631984 9070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b63043ec77940afb4cc3076a804ae9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b63043ec77940afb4cc3076a804ae9d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 44609 } } }
I20260810 06:37:57.632407 9072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:57.632807 9070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:57.634895 9079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:37:57.640127 9079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:37:57.648612 9079 catalog_manager.cc:1347] Generated new cluster ID: 81f979c27b384791ab0302e60f7ea12d
I20260810 06:37:57.648720 9079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:37:57.659461 9079 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:37:57.660917 9079 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:37:57.667817 9079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b63043ec77940afb4cc3076a804ae9d: Generated new TSK 0
I20260810 06:37:57.668592 9079 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:37:57.742609 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:57.687934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51106 (local address 127.31.59.254:44609)
0810 06:37:57.688112 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:57.688118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:57.688452 (+ 334us) server_negotiation.cc:407] Connection header received
0810 06:37:57.688702 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:57.688708 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:57.688795 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:57.688962 (+ 167us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:37:57.689377 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:57.690489 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:57.691860 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:57.692520 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:57.692653 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:57.693135 (+ 482us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:37:57.741727 (+ 48592us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 06:37:57.742205 (+ 478us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51106: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 06:37:57.742653 9109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:37:57.687781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:44609 (local address 127.0.0.1:51106)
0810 06:37:57.688351 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:57.688368 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:57.688508 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:57.688956 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:57.688960 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:57.688973 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:37:57.689227 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:57.689236 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:57.690635 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:57.690640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:57.691721 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:57.691733 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:57.692708 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:57.692714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:57.692836 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:57.692868 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:37:57.742052 (+ 49184us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 06:37:57.742351 (+ 299us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.59.254:44609: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":300,"threads_started":1}
W20260810 06:37:57.742770 9066 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51106: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260810 06:37:57.744417 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 8946
I20260810 06:37:57.745501 9056 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:37:57.838070 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 8946
2026-08-10T06:37:57Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (723 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (1514 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (402 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:37:59 dist-test-slave-4tdj krb5kdc[9136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:37:59 dist-test-slave-4tdj krb5kdc[9136](info): set up 2 sockets
Aug 10 06:37:59 dist-test-slave-4tdj krb5kdc[9136](info): commencing operation
krb5kdc: starting...
W20260810 06:38:00.965296 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.005s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:38:01 dist-test-slave-4tdj krb5kdc[9136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:38:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:38:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:38:01.196388 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:39029
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:39029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:01.519336 9156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:01.519604 9156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:01.519732 9156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:01.526717 9156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:01.526815 9156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:01.526871 9156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:01.526945 9156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:01.527005 9156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:01.536585 9156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:39029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:39029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:01.537855 9156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:01.539115 9156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:01.550194 9167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:01.550252 9170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:01.550274 9168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:01.551386 9156 server_base.cc:1034] running on GCE node
I20260810 06:38:01.552071 9156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:01.553341 9156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:01.554533 9156 hybrid_clock.cc:648] HybridClock initialized: now 1786343881554480 us; error 48 us; skew 500 ppm
I20260810 06:38:01.554848 9156 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:01 dist-test-slave-4tdj krb5kdc[9136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343881, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:01.557763 9156 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:38:01.559265 9156 webserver.cc:460] Webserver started at http://127.31.59.254:46739/ using document root <none> and password file <none>
I20260810 06:38:01.559886 9156 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:01.559986 9156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:01.560252 9156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:01.563159 9156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "4f8e374c1b03460fa2a3f3c5e2c987d7"
format_stamp: "Formatted at 2026-08-10 06:38:01 on dist-test-slave-4tdj"
server_key: "e703aca671a4a2383eec6fe327dc5181"
server_key_iv: "250155b152a7e8ef00c58986c10c2fd2"
server_key_version: "encryptionkey@0"
I20260810 06:38:01.563860 9156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "4f8e374c1b03460fa2a3f3c5e2c987d7"
format_stamp: "Formatted at 2026-08-10 06:38:01 on dist-test-slave-4tdj"
server_key: "e703aca671a4a2383eec6fe327dc5181"
server_key_iv: "250155b152a7e8ef00c58986c10c2fd2"
server_key_version: "encryptionkey@0"
I20260810 06:38:01.568871 9156 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 06:38:01.572167 9181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:01.573405 9156 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:38:01.573539 9156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "4f8e374c1b03460fa2a3f3c5e2c987d7"
format_stamp: "Formatted at 2026-08-10 06:38:01 on dist-test-slave-4tdj"
server_key: "e703aca671a4a2383eec6fe327dc5181"
server_key_iv: "250155b152a7e8ef00c58986c10c2fd2"
server_key_version: "encryptionkey@0"
I20260810 06:38:01.573690 9156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:01.595726 9156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:01.599510 9156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:01.599745 9156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:01.617286 9156 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:39029
I20260810 06:38:01.617328 9264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:39029 every 8 connection(s)
I20260810 06:38:01.618606 9156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:38:01.619367 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 9156
I20260810 06:38:01.619626 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:38:01.620287 31983 external_mini_cluster.cc:1428] Setting key cd29868c5b8e881214c645c90df67bab
I20260810 06:38:01.622695 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:38:01 dist-test-slave-4tdj krb5kdc[9136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:38:01.635623 9265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7: Bootstrap starting.
I20260810 06:38:01.638849 9265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:01.640069 9265 log.cc:824] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:01.643147 9265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7: No bootstrap required, opened a new log
I20260810 06:38:01.643469 9269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:01.622367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47324 (local address 127.31.59.254:39029)
0810 06:38:01.623181 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:01.623193 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:01.623234 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:38:01.624860 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:01.624882 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:01.625287 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:01.625620 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:01.627344 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:01.628729 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:01.630465 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:01.631139 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:01.631268 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:01.636007 (+ 4739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:01.636014 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:01.636027 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:01.636061 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:01.638951 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:01.639769 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:01.639779 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:01.639785 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:01.639943 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:01.640375 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:01.640379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:01.640381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:01.641116 (+ 735us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:01.641409 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:01.642113 (+ 704us) server_negotiation.cc:299] Negotiation successful
0810 06:38:01.642520 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":356,"threads_started":1}
I20260810 06:38:01.651469 9265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8e374c1b03460fa2a3f3c5e2c987d7" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 39029 } }
I20260810 06:38:01.651850 9265 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:01.651947 9265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f8e374c1b03460fa2a3f3c5e2c987d7, State: Initialized, Role: FOLLOWER
I20260810 06:38:01.652532 9265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8e374c1b03460fa2a3f3c5e2c987d7" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 39029 } }
I20260810 06:38:01.652761 9265 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:38:01.652897 9265 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:38:01.653049 9265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:01.655408 9265 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8e374c1b03460fa2a3f3c5e2c987d7" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 39029 } }
I20260810 06:38:01.655889 9265 leader_election.cc:302] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f8e374c1b03460fa2a3f3c5e2c987d7; no voters:
I20260810 06:38:01.656422 9265 leader_election.cc:288] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:38:01.656549 9271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:01.657191 9271 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7 [term 1 LEADER]: Becoming Leader. State: Replica: 4f8e374c1b03460fa2a3f3c5e2c987d7, State: Running, Role: LEADER
I20260810 06:38:01.657757 9271 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8e374c1b03460fa2a3f3c5e2c987d7" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 39029 } }
I20260810 06:38:01.658346 9265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:01.660845 9278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f8e374c1b03460fa2a3f3c5e2c987d7. Latest consensus state: current_term: 1 leader_uuid: "4f8e374c1b03460fa2a3f3c5e2c987d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8e374c1b03460fa2a3f3c5e2c987d7" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 39029 } } }
I20260810 06:38:01.660923 9276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f8e374c1b03460fa2a3f3c5e2c987d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8e374c1b03460fa2a3f3c5e2c987d7" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 39029 } } }
I20260810 06:38:01.661377 9276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:01.662966 9278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:01.663594 9290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:38:01.668097 9290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:38:01.676470 9290 catalog_manager.cc:1347] Generated new cluster ID: ec36f17f7dfb477da5e43f4bc255c896
I20260810 06:38:01.676580 9290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:38:01.691234 9290 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:38:01.692469 9290 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:38:01.700277 9290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f8e374c1b03460fa2a3f3c5e2c987d7: Generated new TSK 0
I20260810 06:38:01.700999 9290 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
I20260810 06:38:01.754457 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:39029
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:02.079002 9310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:02.079268 9310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:02.079361 9310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:02.087199 9310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:02.087313 9310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:02.087430 9310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:38:02.096880 9310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:39029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:02.098039 9310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:02.099370 9310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:02.112002 9315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:02.112010 9321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:02.111933 9316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:02.113051 9310 server_base.cc:1034] running on GCE node
I20260810 06:38:02.113667 9310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:02.114584 9310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:02.115789 9310 hybrid_clock.cc:648] HybridClock initialized: now 1786343882115744 us; error 38 us; skew 500 ppm
I20260810 06:38:02.116084 9310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:02 dist-test-slave-4tdj krb5kdc[9136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343882, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:02.119159 9310 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:38:02.120527 9310 webserver.cc:460] Webserver started at http://127.31.59.193:46847/ using document root <none> and password file <none>
I20260810 06:38:02.121174 9310 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:02.121276 9310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:02.121577 9310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:02.124392 9310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "b0854629799a4a86be2c54c739261994"
format_stamp: "Formatted at 2026-08-10 06:38:02 on dist-test-slave-4tdj"
server_key: "9143fd2d888a3567ae9340646cd71df9"
server_key_iv: "6870c12dbbf71f357a366ee33f0ebb32"
server_key_version: "encryptionkey@0"
I20260810 06:38:02.125149 9310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "b0854629799a4a86be2c54c739261994"
format_stamp: "Formatted at 2026-08-10 06:38:02 on dist-test-slave-4tdj"
server_key: "9143fd2d888a3567ae9340646cd71df9"
server_key_iv: "6870c12dbbf71f357a366ee33f0ebb32"
server_key_version: "encryptionkey@0"
I20260810 06:38:02.130239 9310 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260810 06:38:02.133575 9336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:02.134904 9310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:38:02.135051 9310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "b0854629799a4a86be2c54c739261994"
format_stamp: "Formatted at 2026-08-10 06:38:02 on dist-test-slave-4tdj"
server_key: "9143fd2d888a3567ae9340646cd71df9"
server_key_iv: "6870c12dbbf71f357a366ee33f0ebb32"
server_key_version: "encryptionkey@0"
I20260810 06:38:02.135222 9310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:02.168538 9310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:02.172688 9310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:02.172945 9310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:02.173892 9310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:02.175251 9310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:02.175324 9310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:02.175442 9310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:02.175498 9310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:02.206120 9310 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:39983
I20260810 06:38:02.206163 9536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:39983 every 8 connection(s)
I20260810 06:38:02.207448 9310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:38:02.216161 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 9310
I20260810 06:38:02.216372 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:38:02.216982 31983 external_mini_cluster.cc:1428] Setting key bb69d707a2a01f4d84b96a4e46fd37d3
Aug 10 06:38:02 dist-test-slave-4tdj krb5kdc[9136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343882, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:38:02.227630 9542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:02.209855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:43995 (local address 127.31.59.254:39029)
0810 06:38:02.210254 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:02.210259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:02.211210 (+ 951us) server_negotiation.cc:407] Connection header received
0810 06:38:02.213035 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:02.213041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:02.213124 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:02.213219 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:02.214822 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:02.215529 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:02.217185 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:02.217751 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:02.217881 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:02.221821 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:02.221826 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:02.221830 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:02.221863 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:02.224302 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:02.225180 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:02.225185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:02.225187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:02.225348 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:02.225798 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:02.225801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:02.225803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:02.226107 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:02.226234 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:02.227141 (+ 907us) server_negotiation.cc:299] Negotiation successful
0810 06:38:02.227326 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":204,"threads_started":1}
I20260810 06:38:02.228657 9543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:02.210319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:39029 (local address 127.31.59.193:43995)
0810 06:38:02.211048 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:02.211095 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:02.212752 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:02.213480 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:02.213489 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:02.214012 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:02.214649 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:02.214668 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:02.215681 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:02.215685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:02.217031 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:02.217043 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:02.217998 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:02.218006 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:02.218248 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:02.218904 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:02.218943 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:02.221595 (+ 2652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:02.224441 (+ 2846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:02.224450 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:02.224467 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:02.225008 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:02.225458 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:02.225462 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:02.225465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:02.225697 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:02.226230 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:02.226237 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:02.226681 (+ 444us) client_negotiation.cc:769] Sending connection context
0810 06:38:02.226966 (+ 285us) client_negotiation.cc:240] Negotiation successful
0810 06:38:02.227502 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":276,"threads_started":1}
I20260810 06:38:02.230084 9538 heartbeater.cc:347] Connected to a master server at 127.31.59.254:39029
I20260810 06:38:02.230391 9538 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:02.231097 9538 heartbeater.cc:511] Master 127.31.59.254:39029 requested a full tablet report, sending...
I20260810 06:38:02.233109 9210 ts_manager.cc:194] Registered new tserver with Master: b0854629799a4a86be2c54c739261994 (127.31.59.193:39983)
I20260810 06:38:02.234576 9210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:43995
WARNING: no policy specified for kudu/127.31.59.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
I20260810 06:38:02.266860 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:39029
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:02.583405 9547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:02.583696 9547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:02.583837 9547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:02.590752 9547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:02.590863 9547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:02.591015 9547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:38:02.600240 9547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:39029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:02.601419 9547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:02.602712 9547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:02.614934 9559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:02.614907 9562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:02.615097 9560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:02.615772 9547 server_base.cc:1034] running on GCE node
I20260810 06:38:02.616333 9547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:02.617385 9547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:02.618618 9547 hybrid_clock.cc:648] HybridClock initialized: now 1786343882618522 us; error 81 us; skew 500 ppm
I20260810 06:38:02.618961 9547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:02 dist-test-slave-4tdj krb5kdc[9136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343882, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:02.622596 9547 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:38:02.624125 9547 webserver.cc:460] Webserver started at http://127.31.59.194:42397/ using document root <none> and password file <none>
I20260810 06:38:02.624897 9547 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:02.625039 9547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:02.625388 9547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:02.629128 9547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "ad6eed65bbbe49ccbc2916e969ad9e3b"
format_stamp: "Formatted at 2026-08-10 06:38:02 on dist-test-slave-4tdj"
server_key: "2d8e6ffd4b45f37b2331ee99b32f2d0c"
server_key_iv: "930d67d653315eda14013d12e44ac6c4"
server_key_version: "encryptionkey@0"
I20260810 06:38:02.630066 9547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "ad6eed65bbbe49ccbc2916e969ad9e3b"
format_stamp: "Formatted at 2026-08-10 06:38:02 on dist-test-slave-4tdj"
server_key: "2d8e6ffd4b45f37b2331ee99b32f2d0c"
server_key_iv: "930d67d653315eda14013d12e44ac6c4"
server_key_version: "encryptionkey@0"
I20260810 06:38:02.636309 9547 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.002s
I20260810 06:38:02.639868 9575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:02.641095 9547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260810 06:38:02.641239 9547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "ad6eed65bbbe49ccbc2916e969ad9e3b"
format_stamp: "Formatted at 2026-08-10 06:38:02 on dist-test-slave-4tdj"
server_key: "2d8e6ffd4b45f37b2331ee99b32f2d0c"
server_key_iv: "930d67d653315eda14013d12e44ac6c4"
server_key_version: "encryptionkey@0"
I20260810 06:38:02.641412 9547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:02.658223 9547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:02.662338 9547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:02.662601 9547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:02.663462 9547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:02.664728 9547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:02.664803 9547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:02.664894 9547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:02.664937 9547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:02.694985 9547 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:36847
I20260810 06:38:02.695029 9764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:36847 every 8 connection(s)
I20260810 06:38:02.696204 9547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:38:02.697026 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 9547
I20260810 06:38:02.697224 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:38:02.697687 31983 external_mini_cluster.cc:1428] Setting key 07a445d7616fd951091bc4b399050726
Aug 10 06:38:02 dist-test-slave-4tdj krb5kdc[9136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343882, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:38:02.716843 9542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:02.698783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:46641 (local address 127.31.59.254:39029)
0810 06:38:02.698964 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:02.698980 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:02.700220 (+ 1240us) server_negotiation.cc:407] Connection header received
0810 06:38:02.702046 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:02.702051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:02.702133 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:02.702218 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:02.703692 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:02.704384 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:02.705937 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:02.706395 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:02.706498 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:02.710744 (+ 4246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:02.710751 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:02.710757 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:02.710801 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:02.713625 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:02.714460 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:02.714466 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:02.714470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:02.714659 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:02.715124 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:02.715129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:02.715131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:02.715413 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:02.715516 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:02.716343 (+ 827us) server_negotiation.cc:299] Negotiation successful
0810 06:38:02.716551 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:38:02.719425 9771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:02.699261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:39029 (local address 127.31.59.194:46641)
0810 06:38:02.700056 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:02.700094 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:02.701799 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:02.702479 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:02.702487 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:02.702972 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:02.703525 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:02.703542 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:02.704530 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:02.704534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:02.705799 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:02.705811 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:02.706623 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:02.706631 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:02.706838 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:02.707466 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:02.707502 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:02.710464 (+ 2962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:02.713793 (+ 3329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:02.713803 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:02.713820 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:02.714287 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:02.714787 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:02.714791 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:02.714794 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:02.715015 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:02.715595 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:02.715602 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:02.715981 (+ 379us) client_negotiation.cc:769] Sending connection context
0810 06:38:02.717052 (+ 1071us) client_negotiation.cc:240] Negotiation successful
0810 06:38:02.717705 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":351,"threads_started":1}
I20260810 06:38:02.721119 9767 heartbeater.cc:347] Connected to a master server at 127.31.59.254:39029
I20260810 06:38:02.721415 9767 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:02.722265 9767 heartbeater.cc:511] Master 127.31.59.254:39029 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.59.195@KRBTEST.COM; defaulting to no policy
I20260810 06:38:02.723764 9210 ts_manager.cc:194] Registered new tserver with Master: ad6eed65bbbe49ccbc2916e969ad9e3b (127.31.59.194:36847)
I20260810 06:38:02.724614 9210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:46641
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
I20260810 06:38:02.750823 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:39029
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:03.066926 9776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:03.067186 9776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:03.067284 9776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:03.074201 9776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:03.074324 9776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:03.074462 9776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:38:03.083882 9776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:39029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:03.085028 9776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:03.086269 9776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:03.098603 9790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:03.098780 9784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:03.098627 9786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:03.099321 9776 server_base.cc:1034] running on GCE node
I20260810 06:38:03.099889 9776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:03.100711 9776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:03.101899 9776 hybrid_clock.cc:648] HybridClock initialized: now 1786343883101848 us; error 43 us; skew 500 ppm
I20260810 06:38:03.102190 9776 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:03 dist-test-slave-4tdj krb5kdc[9136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343883, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:03.105160 9776 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:38:03.106559 9776 webserver.cc:460] Webserver started at http://127.31.59.195:41037/ using document root <none> and password file <none>
I20260810 06:38:03.107199 9776 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:03.107308 9776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:03.107589 9776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:03.110349 9776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "aa81c6ed8911498e8d9d0ddcb2eff9fc"
format_stamp: "Formatted at 2026-08-10 06:38:03 on dist-test-slave-4tdj"
server_key: "9ece8519ff5f384ba73d9cdd2949a26e"
server_key_iv: "9366cae2a1a4a21bef5e93717cf08413"
server_key_version: "encryptionkey@0"
I20260810 06:38:03.111099 9776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "aa81c6ed8911498e8d9d0ddcb2eff9fc"
format_stamp: "Formatted at 2026-08-10 06:38:03 on dist-test-slave-4tdj"
server_key: "9ece8519ff5f384ba73d9cdd2949a26e"
server_key_iv: "9366cae2a1a4a21bef5e93717cf08413"
server_key_version: "encryptionkey@0"
I20260810 06:38:03.115914 9776 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260810 06:38:03.119273 9799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:03.120549 9776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:38:03.120689 9776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "aa81c6ed8911498e8d9d0ddcb2eff9fc"
format_stamp: "Formatted at 2026-08-10 06:38:03 on dist-test-slave-4tdj"
server_key: "9ece8519ff5f384ba73d9cdd2949a26e"
server_key_iv: "9366cae2a1a4a21bef5e93717cf08413"
server_key_version: "encryptionkey@0"
I20260810 06:38:03.120867 9776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:03.144915 9776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:03.149019 9776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:03.149286 9776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:03.150223 9776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:03.151535 9776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:03.151604 9776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:03.151718 9776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:03.151768 9776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:03.181253 9776 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:44443
I20260810 06:38:03.181318 9985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:44443 every 8 connection(s)
I20260810 06:38:03.182472 9776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:38:03.189899 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 9776
I20260810 06:38:03.190105 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:38:03.190662 31983 external_mini_cluster.cc:1428] Setting key b4e4af33d57512618d17b6f703638844
Aug 10 06:38:03 dist-test-slave-4tdj krb5kdc[9136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343883, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:38:03.201277 9542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:03.184799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:58379 (local address 127.31.59.254:39029)
0810 06:38:03.184962 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:03.184966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:03.186034 (+ 1068us) server_negotiation.cc:407] Connection header received
0810 06:38:03.187854 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:03.187860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:03.187941 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:03.188035 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:03.189484 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:03.190592 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:03.192187 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:03.192746 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:03.192838 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:03.196126 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:03.196130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:03.196134 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:03.196166 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:03.198714 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:03.199381 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:03.199385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:03.199387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:03.199504 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:03.199903 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:03.199906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:03.199907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:03.200160 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:03.200271 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:03.200864 (+ 593us) server_negotiation.cc:299] Negotiation successful
0810 06:38:03.201023 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:38:03.202164 9991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:03.185162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:39029 (local address 127.31.59.195:58379)
0810 06:38:03.185871 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:03.185913 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:03.187546 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:03.188278 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:03.188286 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:03.188781 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:03.189319 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:03.189336 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:03.190753 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:03.190759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:03.192051 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:03.192063 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:03.192964 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:03.192971 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:03.193267 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:03.193891 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:03.193913 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:03.195925 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:03.198854 (+ 2929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:03.198860 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:03.198870 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:03.199297 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:03.199622 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:03.199626 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:03.199628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:03.199810 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:03.200269 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:03.200275 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:03.200560 (+ 285us) client_negotiation.cc:769] Sending connection context
0810 06:38:03.200783 (+ 223us) client_negotiation.cc:240] Negotiation successful
0810 06:38:03.201134 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":268,"threads_started":1}
I20260810 06:38:03.203390 9987 heartbeater.cc:347] Connected to a master server at 127.31.59.254:39029
I20260810 06:38:03.203694 9987 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:03.204442 9987 heartbeater.cc:511] Master 127.31.59.254:39029 requested a full tablet report, sending...
I20260810 06:38:03.205981 9210 ts_manager.cc:194] Registered new tserver with Master: aa81c6ed8911498e8d9d0ddcb2eff9fc (127.31.59.195:44443)
I20260810 06:38:03.206775 9210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:58379
I20260810 06:38:03.217658 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:38:03.237322 9538 heartbeater.cc:503] Master 127.31.59.254:39029 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/test-admin.keytab.
Aug 10 06:38:03 dist-test-slave-4tdj krb5kdc[9136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:03.245728 31983 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 06:38:03.250351 9998 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:03.248089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44006 (local address 127.31.59.193:39983)
0810 06:38:03.248739 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:03.248748 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:03.248775 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:38:03.248830 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:03.248838 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:03.249045 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:03.249170 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:03.249969 (+ 799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44006: BlockingRecv error: recv got EOF from 127.0.0.1:44006 (error 108)
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":285,"threads_started":1}
W20260810 06:38:03.250422 31983 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/krb5cc' of type FILE
Aug 10 06:38:03 dist-test-slave-4tdj krb5kdc[9136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:03.252067 31983 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 06:38:03 dist-test-slave-4tdj krb5kdc[9136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:03.253297 31983 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 06:38:03 dist-test-slave-4tdj krb5kdc[9136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:38:03.268265 9998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:03.254747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44020 (local address 127.31.59.193:39983)
0810 06:38:03.254936 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:03.254941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:03.255161 (+ 220us) server_negotiation.cc:407] Connection header received
0810 06:38:03.255263 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:03.255266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:03.255327 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:03.255403 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:03.256328 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:03.257332 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:03.258662 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:03.259265 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:03.259368 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:03.262682 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:03.262694 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:03.262708 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:03.262745 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:03.265667 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:03.266360 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:03.266368 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:03.266370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:03.266537 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:03.266889 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:03.266893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:03.266895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:03.267260 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:03.267498 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:03.267784 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 06:38:03.267987 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260810 06:38:03.274001 9998 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:03.272300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44022 (local address 127.31.59.193:39983)
0810 06:38:03.272509 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:03.272514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:03.272799 (+ 285us) server_negotiation.cc:407] Connection header received
0810 06:38:03.272942 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:03.272947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:03.273011 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:03.273087 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:03.273776 (+ 689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44022: BlockingRecv error: recv got EOF from 127.0.0.1:44022 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 06:38:03.274147 31983 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/krb5cc' of type FILE
Aug 10 06:38:03 dist-test-slave-4tdj krb5kdc[9136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:03.275575 31983 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 06:38:03 dist-test-slave-4tdj krb5kdc[9136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:03.276813 31983 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 06:38:03 dist-test-slave-4tdj krb5kdc[9136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:38:03.288945 9998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:03.277980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44032 (local address 127.31.59.193:39983)
0810 06:38:03.278112 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:03.278115 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:03.278394 (+ 279us) server_negotiation.cc:407] Connection header received
0810 06:38:03.278501 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:03.278504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:03.278555 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:03.278627 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:03.279453 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:03.280130 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:03.281223 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:03.281634 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:03.281721 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:03.284837 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:03.284842 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:03.284845 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:03.284874 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:03.287054 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:03.287590 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:03.287594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:03.287596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:03.287712 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:03.287987 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:03.287990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:03.287992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:03.288219 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:03.288391 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:03.288598 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 06:38:03.288739 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260810 06:38:03.292740 9998 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:03.291316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44038 (local address 127.31.59.193:39983)
0810 06:38:03.291488 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:03.291493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:03.291594 (+ 101us) server_negotiation.cc:407] Connection header received
0810 06:38:03.291729 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:03.291735 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:03.291805 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:03.291880 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:03.292534 (+ 654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44038: BlockingRecv error: recv got EOF from 127.0.0.1:44038 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260810 06:38:03.293277 31983 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343831713822-31983-0/krb5kdc/krb5cc)
Aug 10 06:38:03 dist-test-slave-4tdj krb5kdc[9136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:03.294792 31983 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 06:38:03 dist-test-slave-4tdj krb5kdc[9136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:03.296103 31983 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 06:38:03 dist-test-slave-4tdj krb5kdc[9136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:38:03.309933 9998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:03.297342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44040 (local address 127.31.59.193:39983)
0810 06:38:03.297543 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:03.297550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:03.297736 (+ 186us) server_negotiation.cc:407] Connection header received
0810 06:38:03.297883 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:03.297890 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:03.297942 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:03.298039 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:03.299126 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:03.299815 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:03.301082 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:03.301513 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:03.301632 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:03.305072 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:03.305077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:03.305080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:03.305110 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:03.307449 (+ 2339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:03.308080 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:03.308084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:03.308086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:03.308221 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:03.308599 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:03.308603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:03.308605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:03.308869 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:03.309172 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:03.309445 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 06:38:03.309649 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:38:03.311352 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 9310
I20260810 06:38:03.323017 9998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:03.312022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44044 (local address 127.31.59.193:39983)
0810 06:38:03.312205 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:03.312212 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:03.312471 (+ 259us) server_negotiation.cc:407] Connection header received
0810 06:38:03.312674 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:03.312680 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:03.312737 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:03.312816 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:03.313762 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:03.314534 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:03.315741 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:03.316164 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:03.316285 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:03.318275 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:03.318280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:03.318283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:03.318344 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:03.320659 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:03.321254 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:03.321260 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:03.321264 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:03.321396 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:03.321746 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:03.321751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:03.321754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:03.321988 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:03.322226 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:03.322565 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 06:38:03.322747 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:38:03.323485 9531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:03.404181 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 9310
I20260810 06:38:03.428128 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 9547
Aug 10 06:38:03 dist-test-slave-4tdj krb5kdc[9136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.194@KRBTEST.COM
I20260810 06:38:03.443331 10015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:03.429006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52352 (local address 127.31.59.194:36847)
0810 06:38:03.429530 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:03.429537 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:03.429562 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:38:03.429628 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:03.429635 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:03.429831 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:03.429985 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:03.431069 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:03.432105 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:03.433336 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:03.433891 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:03.434015 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:03.437734 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:03.437745 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:03.437758 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:03.437792 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:03.440589 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:03.441252 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:03.441260 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:03.441262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:03.441389 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:03.441750 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:03.441754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:03.441756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:03.442104 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:03.442426 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:03.442784 (+ 358us) server_negotiation.cc:299] Negotiation successful
0810 06:38:03.443041 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":248,"threads_started":1}
I20260810 06:38:03.444716 9754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:03.526945 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 9547
I20260810 06:38:03.549721 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 9776
Aug 10 06:38:03 dist-test-slave-4tdj krb5kdc[9136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.195@KRBTEST.COM
I20260810 06:38:03.565066 10021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:03.550764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58166 (local address 127.31.59.195:44443)
0810 06:38:03.551492 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:03.551499 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:03.551528 (+ 29us) server_negotiation.cc:407] Connection header received
0810 06:38:03.551588 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:03.551595 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:03.551804 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:03.551930 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:03.552951 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:03.553941 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:03.555303 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:03.555835 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:03.555928 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:03.559712 (+ 3784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:03.559723 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:03.559759 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:03.559809 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:03.562471 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:03.563184 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:03.563191 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:03.563193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:03.563317 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:03.563696 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:03.563698 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:03.563700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:03.564013 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:03.564259 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:03.564528 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 06:38:03.564739 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":418,"threads_started":1}
I20260810 06:38:03.566285 9976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:03.648128 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 9776
I20260810 06:38:03.664273 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 9156
I20260810 06:38:03.665421 9256 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:03.765925 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 9156
2026-08-10T06:38:03Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4014 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:38:03 dist-test-slave-4tdj krb5kdc[10030](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:38:03 dist-test-slave-4tdj krb5kdc[10030](info): set up 2 sockets
Aug 10 06:38:03 dist-test-slave-4tdj krb5kdc[10030](info): commencing operation
krb5kdc: starting...
W20260810 06:38:04.975147 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:38:05 dist-test-slave-4tdj krb5kdc[10030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:38:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:38:05Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.254.keytab.
Entry for principal oryx/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.254.keytab.
I20260810 06:38:05.191743 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:36175
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.254.keytab
--principal=oryx/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:38421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:36175
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:05.506916 10052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:05.507175 10052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:05.507300 10052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:05.514086 10052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:05.514210 10052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:05.514355 10052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:05.514444 10052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:05.514500 10052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:05.524362 10052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:38421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:36175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.254.keytab
--principal=oryx/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:36175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:05.525619 10052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:05.526913 10052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:05.537883 10057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:05.537942 10058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:05.537912 10060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:05.539083 10052 server_base.cc:1034] running on GCE node
I20260810 06:38:05.539803 10052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:05.541106 10052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:05.542322 10052 hybrid_clock.cc:648] HybridClock initialized: now 1786343885542257 us; error 32 us; skew 500 ppm
I20260810 06:38:05.542654 10052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:05 dist-test-slave-4tdj krb5kdc[10030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343885, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:05.545646 10052 init.cc:377] Logged in from keytab as oryx/127.31.59.254@KRBTEST.COM (short username oryx)
I20260810 06:38:05.547037 10052 webserver.cc:460] Webserver started at http://127.31.59.254:45621/ using document root <none> and password file <none>
I20260810 06:38:05.547691 10052 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:05.547811 10052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:05.548099 10052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:05.550989 10052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "4f43ba0f328e4a03abc5b1f6d0226f78"
format_stamp: "Formatted at 2026-08-10 06:38:05 on dist-test-slave-4tdj"
server_key: "5dcd8e11a2dd8ef04eb321d803e16904"
server_key_iv: "04742749fddc7285fde77acf2c7ef5ad"
server_key_version: "encryptionkey@0"
I20260810 06:38:05.551716 10052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "4f43ba0f328e4a03abc5b1f6d0226f78"
format_stamp: "Formatted at 2026-08-10 06:38:05 on dist-test-slave-4tdj"
server_key: "5dcd8e11a2dd8ef04eb321d803e16904"
server_key_iv: "04742749fddc7285fde77acf2c7ef5ad"
server_key_version: "encryptionkey@0"
I20260810 06:38:05.556674 10052 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 06:38:05.559922 10073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:05.561163 10052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:38:05.561318 10052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "4f43ba0f328e4a03abc5b1f6d0226f78"
format_stamp: "Formatted at 2026-08-10 06:38:05 on dist-test-slave-4tdj"
server_key: "5dcd8e11a2dd8ef04eb321d803e16904"
server_key_iv: "04742749fddc7285fde77acf2c7ef5ad"
server_key_version: "encryptionkey@0"
I20260810 06:38:05.561486 10052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:05.588526 10052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:05.592414 10052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:05.592662 10052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:05.609777 10052 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:36175
I20260810 06:38:05.609833 10159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:36175 every 8 connection(s)
I20260810 06:38:05.611048 10052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:38:05.614717 10162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:05.620518 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 10052
I20260810 06:38:05.620774 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:38:05.621678 31983 external_mini_cluster.cc:1428] Setting key 77e7a43b88f7a4da64990bf229cb432e
I20260810 06:38:05.626327 10162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78: Bootstrap starting.
I20260810 06:38:05.629472 10162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:05.630544 10162 log.cc:824] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:05.633383 10162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78: No bootstrap required, opened a new log
Aug 10 06:38:05 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM
I20260810 06:38:05.642561 10162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } }
I20260810 06:38:05.642969 10162 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:05.643077 10162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f43ba0f328e4a03abc5b1f6d0226f78, State: Initialized, Role: FOLLOWER
I20260810 06:38:05.643354 10167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:05.623930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52358 (local address 127.31.59.254:36175)
0810 06:38:05.624713 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:05.624724 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:05.624766 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:38:05.626392 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:05.626419 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:05.626801 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:05.627124 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:05.628529 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:05.629816 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:05.631115 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:05.631935 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:05.632140 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:05.636314 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:05.636322 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:05.636335 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:05.636367 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:05.639290 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:05.640081 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:05.640087 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:05.640092 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:05.640253 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:05.640668 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:05.640671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:05.640673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:05.641158 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:05.641379 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:05.641893 (+ 514us) server_negotiation.cc:299] Negotiation successful
0810 06:38:05.642217 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":384,"threads_started":1}
I20260810 06:38:05.643715 10162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } }
I20260810 06:38:05.643980 10162 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:38:05.644122 10162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:38:05.644299 10162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:05.646727 10162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } }
I20260810 06:38:05.647217 10162 leader_election.cc:302] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f43ba0f328e4a03abc5b1f6d0226f78; no voters:
I20260810 06:38:05.647780 10162 leader_election.cc:288] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:38:05.647874 10169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:05.648559 10169 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [term 1 LEADER]: Becoming Leader. State: Replica: 4f43ba0f328e4a03abc5b1f6d0226f78, State: Running, Role: LEADER
I20260810 06:38:05.649063 10169 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } }
I20260810 06:38:05.649930 10162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:05.651788 10172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f43ba0f328e4a03abc5b1f6d0226f78. Latest consensus state: current_term: 1 leader_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } }
I20260810 06:38:05.651798 10170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } }
I20260810 06:38:05.652262 10172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:05.652262 10170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:05.662734 10178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:38:05.667611 10178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:38:05.676155 10178 catalog_manager.cc:1347] Generated new cluster ID: 1a7d2e9180204d73b144bbdde26e2e50
I20260810 06:38:05.676263 10178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:38:05.687430 10178 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:38:05.689165 10178 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:38:05.696250 10178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78: Generated new TSK 0
I20260810 06:38:05.697172 10178 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.193.keytab.
Entry for principal oryx/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.193.keytab.
I20260810 06:38:05.757849 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36175
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.193.keytab
--principal=oryx/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:38421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:06.076874 10202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:06.077131 10202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:06.077225 10202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:06.084048 10202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:06.084147 10202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:06.084264 10202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:38:06.093277 10202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:38421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.193.keytab
--principal=oryx/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:06.094399 10202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:06.095606 10202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:06.108142 10212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:06.108469 10218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:06.108446 10213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:06.108945 10202 server_base.cc:1034] running on GCE node
I20260810 06:38:06.109496 10202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:06.110351 10202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:06.111582 10202 hybrid_clock.cc:648] HybridClock initialized: now 1786343886111523 us; error 49 us; skew 500 ppm
I20260810 06:38:06.111871 10202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:06 dist-test-slave-4tdj krb5kdc[10030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343886, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:06.114915 10202 init.cc:377] Logged in from keytab as oryx/127.31.59.193@KRBTEST.COM (short username oryx)
I20260810 06:38:06.116242 10202 webserver.cc:460] Webserver started at http://127.31.59.193:45543/ using document root <none> and password file <none>
I20260810 06:38:06.116888 10202 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:06.117005 10202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:06.117277 10202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:06.120085 10202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "7862319cb6c14028878f112d67842e80"
format_stamp: "Formatted at 2026-08-10 06:38:06 on dist-test-slave-4tdj"
server_key: "8485d1d4f0cb8a3d4ef370e94b462d81"
server_key_iv: "a354a3e0ad76676618f57ccc1ad9930d"
server_key_version: "encryptionkey@0"
I20260810 06:38:06.120834 10202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "7862319cb6c14028878f112d67842e80"
format_stamp: "Formatted at 2026-08-10 06:38:06 on dist-test-slave-4tdj"
server_key: "8485d1d4f0cb8a3d4ef370e94b462d81"
server_key_iv: "a354a3e0ad76676618f57ccc1ad9930d"
server_key_version: "encryptionkey@0"
I20260810 06:38:06.125659 10202 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260810 06:38:06.128906 10231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:06.130096 10202 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:38:06.130250 10202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "7862319cb6c14028878f112d67842e80"
format_stamp: "Formatted at 2026-08-10 06:38:06 on dist-test-slave-4tdj"
server_key: "8485d1d4f0cb8a3d4ef370e94b462d81"
server_key_iv: "a354a3e0ad76676618f57ccc1ad9930d"
server_key_version: "encryptionkey@0"
I20260810 06:38:06.130473 10202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:06.144328 10202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:06.150817 10202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:06.151062 10202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:06.151954 10202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:06.153184 10202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:06.153261 10202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:06.153374 10202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:06.153431 10202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:06.182701 10202 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:33401
I20260810 06:38:06.182765 10433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:33401 every 8 connection(s)
I20260810 06:38:06.183933 10202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:38:06.187497 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 10202
I20260810 06:38:06.187716 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:38:06.188397 31983 external_mini_cluster.cc:1428] Setting key aeaffbfedae1a01764d95ac3616c07ab
Aug 10 06:38:06 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343886, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM
I20260810 06:38:06.203910 10439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:06.186418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:44255 (local address 127.31.59.254:36175)
0810 06:38:06.186859 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:06.186864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:06.187963 (+ 1099us) server_negotiation.cc:407] Connection header received
0810 06:38:06.189843 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:06.189849 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:06.189924 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:06.190017 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:06.191537 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:06.192274 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:06.193777 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:06.194182 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:06.194316 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:06.197906 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:06.197911 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:06.197923 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:06.197960 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:06.201014 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:06.201688 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:06.201693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:06.201696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:06.201841 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:06.202225 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:06.202229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:06.202231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:06.202589 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:06.202726 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:06.203334 (+ 608us) server_negotiation.cc:299] Negotiation successful
0810 06:38:06.203563 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":215,"threads_started":1}
I20260810 06:38:06.204566 10441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:06.186977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.193:44255)
0810 06:38:06.187785 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:06.187831 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:06.189571 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:06.190270 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:06.190278 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:06.190804 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:06.191364 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:06.191382 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:06.192429 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:06.192433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:06.193638 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:06.193650 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:06.194420 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:06.194429 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:06.194666 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:06.195270 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:06.195293 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:06.197674 (+ 2381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:06.201150 (+ 3476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:06.201156 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:06.201168 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:06.201558 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:06.201935 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:06.201938 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:06.201941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:06.202114 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:06.202692 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:06.202698 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:06.202994 (+ 296us) client_negotiation.cc:769] Sending connection context
0810 06:38:06.203197 (+ 203us) client_negotiation.cc:240] Negotiation successful
0810 06:38:06.203551 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":323,"threads_started":1}
I20260810 06:38:06.205989 10435 heartbeater.cc:347] Connected to a master server at 127.31.59.254:36175
I20260810 06:38:06.206324 10435 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:06.207190 10435 heartbeater.cc:511] Master 127.31.59.254:36175 requested a full tablet report, sending...
I20260810 06:38:06.209477 10107 ts_manager.cc:194] Registered new tserver with Master: 7862319cb6c14028878f112d67842e80 (127.31.59.193:33401)
I20260810 06:38:06.210997 10107 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:44255
WARNING: no policy specified for oryx/127.31.59.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.194.keytab.
Entry for principal oryx/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.194.keytab.
I20260810 06:38:06.238883 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36175
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.194.keytab
--principal=oryx/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:38421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:06.549690 10444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:06.549955 10444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:06.550060 10444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:06.557092 10444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:06.557191 10444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:06.557303 10444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:38:06.566606 10444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:38421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.194.keytab
--principal=oryx/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:06.567739 10444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:06.568959 10444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:06.581343 10450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:06.581382 10453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:06.581301 10455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:06.582041 10444 server_base.cc:1034] running on GCE node
I20260810 06:38:06.582653 10444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:06.583493 10444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:06.584704 10444 hybrid_clock.cc:648] HybridClock initialized: now 1786343886584652 us; error 36 us; skew 500 ppm
I20260810 06:38:06.585001 10444 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:06 dist-test-slave-4tdj krb5kdc[10030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343886, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:06.587945 10444 init.cc:377] Logged in from keytab as oryx/127.31.59.194@KRBTEST.COM (short username oryx)
I20260810 06:38:06.589277 10444 webserver.cc:460] Webserver started at http://127.31.59.194:45857/ using document root <none> and password file <none>
I20260810 06:38:06.589887 10444 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:06.589998 10444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:06.590271 10444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:06.592980 10444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "3fa769dc3836400cb8b62f521a3aad4a"
format_stamp: "Formatted at 2026-08-10 06:38:06 on dist-test-slave-4tdj"
server_key: "315565a7ce08fd5b8690795f4188e08b"
server_key_iv: "fbbf2f45391d5e8e7256863f6e248ada"
server_key_version: "encryptionkey@0"
I20260810 06:38:06.593704 10444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "3fa769dc3836400cb8b62f521a3aad4a"
format_stamp: "Formatted at 2026-08-10 06:38:06 on dist-test-slave-4tdj"
server_key: "315565a7ce08fd5b8690795f4188e08b"
server_key_iv: "fbbf2f45391d5e8e7256863f6e248ada"
server_key_version: "encryptionkey@0"
I20260810 06:38:06.598420 10444 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260810 06:38:06.601567 10467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:06.602836 10444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:38:06.602986 10444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "3fa769dc3836400cb8b62f521a3aad4a"
format_stamp: "Formatted at 2026-08-10 06:38:06 on dist-test-slave-4tdj"
server_key: "315565a7ce08fd5b8690795f4188e08b"
server_key_iv: "fbbf2f45391d5e8e7256863f6e248ada"
server_key_version: "encryptionkey@0"
I20260810 06:38:06.603156 10444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:06.623222 10444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:06.626821 10444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:06.627059 10444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:06.627934 10444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:06.629143 10444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:06.629212 10444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:06.629326 10444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:06.629375 10444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:06.660444 10444 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:36093
I20260810 06:38:06.660481 10654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:36093 every 8 connection(s)
I20260810 06:38:06.661796 10444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:38:06.666908 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 10444
I20260810 06:38:06.667109 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:38:06.667560 31983 external_mini_cluster.cc:1428] Setting key 1b7f4f8de422d771acba53756ba2caa1
Aug 10 06:38:06 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343886, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM
I20260810 06:38:06.682246 10439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:06.664382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:50785 (local address 127.31.59.254:36175)
0810 06:38:06.664582 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:06.664587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:06.665801 (+ 1214us) server_negotiation.cc:407] Connection header received
0810 06:38:06.667698 (+ 1897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:06.667705 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:06.667798 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:06.667923 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:06.669699 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:06.670801 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:06.672313 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:06.672927 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:06.673066 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:06.676477 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:06.676484 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:06.676488 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:06.676529 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:06.679264 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:06.680036 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:06.680040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:06.680042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:06.680175 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:06.680602 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:06.680605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:06.680607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:06.680852 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:06.680965 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:06.681820 (+ 855us) server_negotiation.cc:299] Negotiation successful
0810 06:38:06.681991 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:38:06.683544 10660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:06.664881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.194:50785)
0810 06:38:06.665645 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:06.665683 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:06.667420 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:06.668126 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:06.668134 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:06.668756 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:06.669530 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:06.669555 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:06.670952 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:06.670956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:06.672160 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:06.672172 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:06.673131 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:06.673135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:06.673321 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:06.673986 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:06.674014 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:06.676252 (+ 2238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:06.679400 (+ 3148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:06.679409 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:06.679425 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:06.679933 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:06.680272 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:06.680276 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:06.680279 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:06.680503 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:06.680971 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:06.680978 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:06.681375 (+ 397us) client_negotiation.cc:769] Sending connection context
0810 06:38:06.681699 (+ 324us) client_negotiation.cc:240] Negotiation successful
0810 06:38:06.682197 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":304,"threads_started":1}
I20260810 06:38:06.684855 10655 heartbeater.cc:347] Connected to a master server at 127.31.59.254:36175
I20260810 06:38:06.685160 10655 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:06.686033 10655 heartbeater.cc:511] Master 127.31.59.254:36175 requested a full tablet report, sending...
I20260810 06:38:06.687532 10107 ts_manager.cc:194] Registered new tserver with Master: 3fa769dc3836400cb8b62f521a3aad4a (127.31.59.194:36093)
I20260810 06:38:06.688341 10107 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:50785
WARNING: no policy specified for oryx/127.31.59.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.195.keytab.
Entry for principal oryx/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.195.keytab.
I20260810 06:38:06.717449 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36175
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.195.keytab
--principal=oryx/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:38421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:07.036267 10664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:07.036571 10664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:07.036684 10664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:07.044013 10664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:07.044135 10664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:07.044261 10664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:38:07.053865 10664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:38421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.195.keytab
--principal=oryx/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:07.055105 10664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:07.056393 10664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:07.068710 10679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:07.068742 10680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:07.068778 10683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:07.069509 10664 server_base.cc:1034] running on GCE node
I20260810 06:38:07.070048 10664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:07.070941 10664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:07.072132 10664 hybrid_clock.cc:648] HybridClock initialized: now 1786343887072081 us; error 45 us; skew 500 ppm
I20260810 06:38:07.072414 10664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:07 dist-test-slave-4tdj krb5kdc[10030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343887, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:07.075554 10664 init.cc:377] Logged in from keytab as oryx/127.31.59.195@KRBTEST.COM (short username oryx)
I20260810 06:38:07.077836 10664 webserver.cc:460] Webserver started at http://127.31.59.195:34305/ using document root <none> and password file <none>
I20260810 06:38:07.078543 10664 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:07.078646 10664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:07.078889 10664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:07.081643 10664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "39dafaff7626463d9a8995dfdcdc2e35"
format_stamp: "Formatted at 2026-08-10 06:38:07 on dist-test-slave-4tdj"
server_key: "a53cce225b5e8287b37b472e1027a3e4"
server_key_iv: "a9abc20920a4501e1b239c20e1ffd585"
server_key_version: "encryptionkey@0"
I20260810 06:38:07.082419 10664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "39dafaff7626463d9a8995dfdcdc2e35"
format_stamp: "Formatted at 2026-08-10 06:38:07 on dist-test-slave-4tdj"
server_key: "a53cce225b5e8287b37b472e1027a3e4"
server_key_iv: "a9abc20920a4501e1b239c20e1ffd585"
server_key_version: "encryptionkey@0"
I20260810 06:38:07.087281 10664 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 06:38:07.090474 10699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:07.091876 10664 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:38:07.092037 10664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "39dafaff7626463d9a8995dfdcdc2e35"
format_stamp: "Formatted at 2026-08-10 06:38:07 on dist-test-slave-4tdj"
server_key: "a53cce225b5e8287b37b472e1027a3e4"
server_key_iv: "a9abc20920a4501e1b239c20e1ffd585"
server_key_version: "encryptionkey@0"
I20260810 06:38:07.092221 10664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:07.113780 10664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:07.117699 10664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:07.117926 10664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:07.118827 10664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:07.120033 10664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:07.120105 10664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:07.120193 10664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:07.120235 10664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:07.149293 10664 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:44247
I20260810 06:38:07.149349 10893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:44247 every 8 connection(s)
I20260810 06:38:07.150557 10664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:38:07.156452 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 10664
I20260810 06:38:07.156653 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:38:07.157060 31983 external_mini_cluster.cc:1428] Setting key 8f16e4087174a8ad99516d043a0d89ce
Aug 10 06:38:07 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343887, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM
I20260810 06:38:07.170730 10439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:07.152972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:41481 (local address 127.31.59.254:36175)
0810 06:38:07.153182 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:07.153188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:07.154258 (+ 1070us) server_negotiation.cc:407] Connection header received
0810 06:38:07.156079 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:07.156085 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:07.156180 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:07.156288 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:07.158000 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.159230 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.161503 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.161913 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.162033 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:07.165758 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:07.165763 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:07.165766 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:07.165798 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:07.168345 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:07.168953 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:07.168956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:07.168959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:07.169083 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:07.169397 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:07.169400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:07.169401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:07.169641 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:07.169746 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:07.170279 (+ 533us) server_negotiation.cc:299] Negotiation successful
0810 06:38:07.170468 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:38:07.171905 10897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:07.153416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.195:41481)
0810 06:38:07.154099 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:07.154142 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:07.155848 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:07.156482 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:07.156491 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:07.157061 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:07.157759 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:07.157793 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.160010 (+ 2217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.160016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:07.161365 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:07.161378 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.162123 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.162130 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:07.162413 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:07.163067 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:07.163089 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:07.165530 (+ 2441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:07.168459 (+ 2929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:07.168466 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:07.168475 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:07.168863 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:07.169159 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:07.169162 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:07.169164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:07.169325 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:07.169730 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:07.169735 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:07.170007 (+ 272us) client_negotiation.cc:769] Sending connection context
0810 06:38:07.170219 (+ 212us) client_negotiation.cc:240] Negotiation successful
0810 06:38:07.170632 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":258,"threads_started":1}
I20260810 06:38:07.173071 10894 heartbeater.cc:347] Connected to a master server at 127.31.59.254:36175
I20260810 06:38:07.173333 10894 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:07.173966 10894 heartbeater.cc:511] Master 127.31.59.254:36175 requested a full tablet report, sending...
I20260810 06:38:07.175277 10107 ts_manager.cc:194] Registered new tserver with Master: 39dafaff7626463d9a8995dfdcdc2e35 (127.31.59.195:44247)
I20260810 06:38:07.176031 10107 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:41481
I20260810 06:38:07.184710 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.59.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.253@KRBTEST.COM" created.
I20260810 06:38:07.213838 10435 heartbeater.cc:503] Master 127.31.59.254:36175 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.253.keytab.
Entry for principal oryx/127.31.59.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.253.keytab.
I20260810 06:38:07.233201 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.253:37507
--webserver_interface=127.31.59.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.253.keytab
--principal=oryx/127.31.59.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:38421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:36175,127.31.59.253:37507
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:07.553161 10902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:07.553435 10902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:07.553532 10902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:07.560637 10902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:07.560755 10902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:07.560815 10902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:07.560883 10902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:07.560927 10902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:07.570744 10902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:38421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:36175,127.31.59.253:37507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.253.keytab
--principal=oryx/127.31.59.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.253:37507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.59.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
W20260810 06:38:07.571060 10902 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.31.59.254:36175,127.31.59.253:37507'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 06:38:07.572016 10902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:07.573271 10902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:07.584301 10909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:07.584302 10913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:07.584345 10915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:07.585338 10902 server_base.cc:1034] running on GCE node
I20260810 06:38:07.586117 10902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:07.587430 10902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:07.588652 10902 hybrid_clock.cc:648] HybridClock initialized: now 1786343887588575 us; error 44 us; skew 500 ppm
I20260810 06:38:07.588930 10902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:07 dist-test-slave-4tdj krb5kdc[10030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343887, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:07.592229 10902 init.cc:377] Logged in from keytab as oryx/127.31.59.253@KRBTEST.COM (short username oryx)
I20260810 06:38:07.593729 10902 webserver.cc:460] Webserver started at http://127.31.59.253:34013/ using document root <none> and password file <none>
I20260810 06:38:07.594442 10902 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:07.594553 10902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:07.594856 10902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:07.597780 10902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/data/instance:
uuid: "a63e2108523a4d96ac09190af21127a0"
format_stamp: "Formatted at 2026-08-10 06:38:07 on dist-test-slave-4tdj"
server_key: "a7a868195adefd3b8ad503de0db1a719"
server_key_iv: "b2313673f0169eb3bdb903233fd6ce96"
server_key_version: "encryptionkey@0"
I20260810 06:38:07.598603 10902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/wal/instance:
uuid: "a63e2108523a4d96ac09190af21127a0"
format_stamp: "Formatted at 2026-08-10 06:38:07 on dist-test-slave-4tdj"
server_key: "a7a868195adefd3b8ad503de0db1a719"
server_key_iv: "b2313673f0169eb3bdb903233fd6ce96"
server_key_version: "encryptionkey@0"
I20260810 06:38:07.603504 10902 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:38:07.606832 10928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:07.608182 10902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:38:07.608337 10902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/wal
uuid: "a63e2108523a4d96ac09190af21127a0"
format_stamp: "Formatted at 2026-08-10 06:38:07 on dist-test-slave-4tdj"
server_key: "a7a868195adefd3b8ad503de0db1a719"
server_key_iv: "b2313673f0169eb3bdb903233fd6ce96"
server_key_version: "encryptionkey@0"
I20260810 06:38:07.608516 10902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:07.644515 10902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:07.648664 10902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:07.648916 10902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:07.667006 10902 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.253:37507
I20260810 06:38:07.666980 11013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.253:37507 every 8 connection(s)
I20260810 06:38:07.668290 10902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/data/info.pb
I20260810 06:38:07.671844 11015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:07.673195 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 10902
I20260810 06:38:07.680542 11015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:38:07 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.253@KRBTEST.COM
Aug 10 06:38:07 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343887, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.253@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM
I20260810 06:38:07.691330 10655 heartbeater.cc:503] Master 127.31.59.254:36175 was elected leader, sending a full tablet report...
I20260810 06:38:07.695364 11020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:07.675504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52122 (local address 127.31.59.253:37507)
0810 06:38:07.676255 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:07.676267 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:07.676305 (+ 38us) server_negotiation.cc:407] Connection header received
0810 06:38:07.677973 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:07.677994 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:07.678409 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:07.678763 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:07.680215 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.681580 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.683108 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.684083 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.684250 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:07.688329 (+ 4079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:07.688337 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:07.688347 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:07.688389 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:07.691434 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:07.692155 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:07.692163 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:07.692171 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:07.692355 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:07.692750 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:07.692755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:07.692758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:07.693288 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:07.693563 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:07.693980 (+ 417us) server_negotiation.cc:299] Negotiation successful
0810 06:38:07.694391 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":302,"threads_started":1}
I20260810 06:38:07.697404 11023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:07.682938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.0.0.1:52362)
0810 06:38:07.683634 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:07.683680 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:07.683839 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:07.684187 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:07.684194 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:07.684854 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:07.685139 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:07.685148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.686208 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.686211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:07.687529 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:07.687543 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.688293 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.688300 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:07.688456 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:07.689014 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:07.689037 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:07.691553 (+ 2516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:07.694867 (+ 3314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:07.694878 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:07.694885 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:07.695302 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:07.695701 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:07.695705 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:07.695709 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:07.695879 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:07.696410 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:07.696479 (+ 69us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:07.696716 (+ 237us) client_negotiation.cc:769] Sending connection context
0810 06:38:07.696895 (+ 179us) client_negotiation.cc:240] Negotiation successful
0810 06:38:07.697129 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":373,"threads_started":1}
I20260810 06:38:07.697855 11022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:07.682868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52362 (local address 127.31.59.254:36175)
0810 06:38:07.683311 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:07.683318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:07.683775 (+ 457us) server_negotiation.cc:407] Connection header received
0810 06:38:07.683952 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:07.683956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:07.684018 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:07.684114 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:07.685294 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.686086 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.687670 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.688075 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.688183 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:07.691703 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:07.691707 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:07.691710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:07.691743 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:07.694752 (+ 3009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:07.695421 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:07.695426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:07.695430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:07.695584 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:07.695988 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:07.695991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:07.695993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:07.696291 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:07.696421 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:07.697016 (+ 595us) server_negotiation.cc:299] Negotiation successful
0810 06:38:07.697233 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":231,"threads_started":1}
I20260810 06:38:07.699383 11015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:38:07 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343887, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.253@KRBTEST.COM for oryx/127.31.59.253@KRBTEST.COM
I20260810 06:38:07.710131 11023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:07.700034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:37507 (local address 127.0.0.1:52126)
0810 06:38:07.700188 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:07.700203 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:07.700290 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:07.700534 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:07.700538 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:07.700853 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:07.700968 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:07.700975 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.701865 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.701868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:07.702717 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:07.702728 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.703340 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.703345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:07.703494 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:07.703897 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:07.703911 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:07.705760 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:07.708312 (+ 2552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:07.708315 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:07.708318 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:07.708675 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:07.708941 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:07.708944 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:07.708947 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:07.709066 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:07.709497 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:07.709500 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:07.709619 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 06:38:07.709721 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 06:38:07.709901 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 06:38:07.710325 11020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:07.700204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52126 (local address 127.31.59.253:37507)
0810 06:38:07.700333 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:07.700338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:07.700353 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:38:07.700389 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:07.700392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:07.700434 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:07.700503 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:07.701091 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.701739 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.702833 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.703218 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.703320 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:07.705922 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:07.705927 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:07.705930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:07.705957 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:07.708208 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:07.708757 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:07.708761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:07.708763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:07.708866 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:07.709149 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:07.709152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:07.709154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:07.709406 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:07.709504 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:07.709786 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 06:38:07.710005 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":40}
I20260810 06:38:07.714718 11015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: Bootstrap starting.
I20260810 06:38:07.717680 11015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:07.718705 11015 log.cc:824] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:07.721226 11015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: No bootstrap required, opened a new log
I20260810 06:38:07.731439 11015 raft_consensus.cc:348] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } }
I20260810 06:38:07.731810 11015 raft_consensus.cc:374] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:07.731915 11015 raft_consensus.cc:729] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a63e2108523a4d96ac09190af21127a0, State: Initialized, Role: FOLLOWER
I20260810 06:38:07.732544 11015 consensus_queue.cc:260] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } }
I20260810 06:38:07.733485 11025 sys_catalog.cc:455] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } } }
I20260810 06:38:07.733826 11025 sys_catalog.cc:458] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:38:07.734858 11015 sys_catalog.cc:564] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 06:38:07.743207 11042 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:38:07.743321 11042 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:38:07.746969 10902 master_runner.cc:428] Detected that this master a63e2108523a4d96ac09190af21127a0 is joining an existing cluster
I20260810 06:38:07.747058 10902 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.59.253:37507
I20260810 06:38:07.770541 11020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:07.755059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43688 (local address 127.31.59.253:37507)
0810 06:38:07.755223 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:07.755227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:07.755765 (+ 538us) server_negotiation.cc:407] Connection header received
0810 06:38:07.755934 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:07.755939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:07.756027 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:07.756133 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:07.757555 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.758729 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.760680 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.761402 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.761495 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:07.764159 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:07.764168 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:07.764173 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:07.764204 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:07.767730 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:07.768440 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:07.768444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:07.768447 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:07.768593 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:07.768930 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:07.768934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:07.768938 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:07.769371 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:07.769517 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:07.769971 (+ 454us) server_negotiation.cc:299] Negotiation successful
0810 06:38:07.770169 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:38:07.771746 11049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:07.754867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:37507 (local address 127.0.0.1:43688)
0810 06:38:07.755662 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:07.755680 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:07.755811 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:07.756292 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:07.756296 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:07.757013 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:07.757394 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:07.757407 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.758944 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.758949 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:07.760541 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:07.760556 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.761786 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.761792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:07.761964 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:07.762814 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:07.762838 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:07.763973 (+ 1135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:07.767853 (+ 3880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:07.767857 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:07.767862 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:07.768336 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:07.768703 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:07.768706 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:07.768709 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:07.768840 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:07.769591 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:07.769595 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:07.769741 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 06:38:07.771068 (+ 1327us) client_negotiation.cc:240] Negotiation successful
0810 06:38:07.771340 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":465,"threads_started":1}
WARNING: no policy specified for oryx/127.31.59.252@KRBTEST.COM; defaulting to no policy
I20260810 06:38:07.779661 11022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:07.754771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47264 (local address 127.31.59.254:36175)
0810 06:38:07.754956 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:07.754962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:07.763180 (+ 8218us) server_negotiation.cc:407] Connection header received
0810 06:38:07.763370 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:07.763376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:07.763468 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:07.763549 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:07.765294 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.766685 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.768712 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.769343 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.769442 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:07.771822 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:07.771828 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:07.771833 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:07.771871 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:07.776356 (+ 4485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:07.777114 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:07.777120 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:07.777123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:07.777292 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:07.777780 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:07.777786 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:07.777789 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:07.778167 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:07.778371 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:07.778984 (+ 613us) server_negotiation.cc:299] Negotiation successful
0810 06:38:07.779236 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:38:07.779646 11050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:07.754879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.0.0.1:47264)
0810 06:38:07.763079 (+ 8200us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:07.763095 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:07.763211 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:07.763738 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:07.763742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:07.764439 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:07.764785 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:07.764801 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.766899 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.766904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:07.768541 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:07.768560 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.769738 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.769745 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:07.769925 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:07.770510 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:07.770545 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:07.771601 (+ 1056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:07.776506 (+ 4905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:07.776511 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:07.776515 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:07.776982 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:07.777435 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:07.777440 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:07.777444 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:07.777631 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:07.778430 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:07.778435 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:07.778676 (+ 241us) client_negotiation.cc:769] Sending connection context
0810 06:38:07.778850 (+ 174us) client_negotiation.cc:240] Negotiation successful
0810 06:38:07.779209 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8050,"mutex_wait_us":1037,"thread_start_us":7267,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.252@KRBTEST.COM" created.
Aug 10 06:38:07 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343885, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.254@KRBTEST.COM for oryx/127.31.59.253@KRBTEST.COM
I20260810 06:38:07.808238 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:07.786476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:37507 (local address 127.0.0.1:43700)
0810 06:38:07.791415 (+ 4939us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:07.791468 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:07.791925 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:07.792428 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:07.792437 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:07.793295 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:07.793582 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:07.793596 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.795008 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.795022 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:07.796693 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:07.796709 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.797693 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.797708 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:07.797921 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:07.798632 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:07.798662 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:07.801466 (+ 2804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:07.805546 (+ 4080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:07.805557 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:07.805563 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:07.805928 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:07.806418 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:07.806422 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:07.806424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:07.806607 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:07.807255 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:07.807261 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:07.807433 (+ 172us) client_negotiation.cc:769] Sending connection context
0810 06:38:07.807622 (+ 189us) client_negotiation.cc:240] Negotiation successful
0810 06:38:07.807891 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4678,"thread_start_us":4511,"threads_started":1}
I20260810 06:38:07.808467 11020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:07.786500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43700 (local address 127.31.59.253:37507)
0810 06:38:07.786666 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:07.786672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:07.791760 (+ 5088us) server_negotiation.cc:407] Connection header received
0810 06:38:07.792149 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:07.792154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:07.792240 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:07.792339 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:07.793761 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.794868 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.796839 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.797498 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.797618 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:07.801668 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:07.801675 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:07.801681 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:07.801718 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:07.805360 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:07.806076 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:07.806082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:07.806086 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:07.806277 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:07.806723 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:07.806728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:07.806732 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:07.807129 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:07.807275 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:07.807741 (+ 466us) server_negotiation.cc:299] Negotiation successful
0810 06:38:07.807998 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.252.keytab.
Entry for principal oryx/127.31.59.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.252.keytab.
I20260810 06:38:07.810395 10107 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.59.253:37507: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a63e2108523a4d96ac09190af21127a0"
member_type: NON_VOTER
last_known_addr {
host: "127.31.59.253"
port: 37507
}
attrs {
promote: true
}
}
dest_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78"
cas_config_opid_index: -1
I20260810 06:38:07.811882 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.252:37537
--webserver_interface=127.31.59.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.252.keytab
--principal=oryx/127.31.59.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:38421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:36175,127.31.59.253:37507,127.31.59.252:37537
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:38:07.812047 10107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: NON_VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: true } }
I20260810 06:38:07.822075 10969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:07.821982 11058 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER a63e2108523a4d96ac09190af21127a0 (127.31.59.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: NON_VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: true } } }
I20260810 06:38:07.825786 10969 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 1 FOLLOWER]: Refusing update from remote peer 4f43ba0f328e4a03abc5b1f6d0226f78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 06:38:07.826350 11058 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.59.253:37507
I20260810 06:38:07.826936 11058 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: NON_VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:07.827359 10902 master.cc:537] Master@127.31.59.253:37507 shutting down...
I20260810 06:38:07.828651 11056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: NON_VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: true } } }
I20260810 06:38:07.829046 11056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: This master's current role is: LEADER
W20260810 06:38:07.841697 10079 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260810 06:38:07.846487 10902 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:07.847185 10902 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:07.847867 10902 tablet_replica.cc:330] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: stopping tablet replica
I20260810 06:38:07.859452 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:07.858963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:37507 (local address 127.0.0.1:43706)
0810 06:38:07.859126 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:07.859252 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:37507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 06:38:07.859671 10079 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 -> Peer a63e2108523a4d96ac09190af21127a0 (127.31.59.253:37507): Couldn't send request to peer a63e2108523a4d96ac09190af21127a0. Status: Network error: Client connection negotiation failed: client connection to 127.31.59.253:37507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 06:38:07.874118 10902 master.cc:552] Master@127.31.59.253:37507 shutdown complete.
I20260810 06:38:07.874274 10902 master_runner.cc:305] Clearing existing system tablet
I20260810 06:38:07.874840 10902 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:07.874954 10902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:07.877980 10902 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260810 06:38:07.879738 11068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:07.880096 10902 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:07.880401 10902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:38:07.880501 10902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/wal
uuid: "a63e2108523a4d96ac09190af21127a0"
format_stamp: "Formatted at 2026-08-10 06:38:07 on dist-test-slave-4tdj"
server_key: "a7a868195adefd3b8ad503de0db1a719"
server_key_iv: "b2313673f0169eb3bdb903233fd6ce96"
server_key_version: "encryptionkey@0"
I20260810 06:38:07.886180 10902 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:07.893069 10902 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 06:38:07.893261 10902 log.cc:1196] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 06:38:07.893726 10902 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: Deleting consensus metadata
I20260810 06:38:07.893994 10902 master_runner.cc:315] Copying system tablet from 127.31.59.254:36175
I20260810 06:38:07.896957 10902 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: tablet copy: Beginning tablet copy session from remote peer at address 127.31.59.254:36175
I20260810 06:38:07.917953 11022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:07.898086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47266 (local address 127.31.59.254:36175)
0810 06:38:07.898274 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:07.898281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:07.902820 (+ 4539us) server_negotiation.cc:407] Connection header received
0810 06:38:07.903033 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:07.903039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:07.903134 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:07.903247 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:07.904406 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.905569 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.907997 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.908689 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.908816 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:07.911247 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:07.911254 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:07.911259 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:07.911302 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:07.914807 (+ 3505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:07.915665 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:07.915670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:07.915674 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:07.915831 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:07.916298 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:07.916303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:07.916307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:07.916666 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:07.916822 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:07.917335 (+ 513us) server_negotiation.cc:299] Negotiation successful
0810 06:38:07.917574 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 06:38:07.917991 11076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:07.898053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.0.0.1:47266)
0810 06:38:07.902663 (+ 4610us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:07.902691 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:07.902871 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:07.903302 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:07.903307 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:07.903915 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:07.904226 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:07.904240 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.905745 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.905751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:07.907834 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:07.907850 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:07.908884 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:07.908891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:07.909083 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:07.909820 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:07.909844 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:07.911020 (+ 1176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:07.914949 (+ 3929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:07.914954 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:07.914959 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:07.915501 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:07.915963 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:07.915973 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:07.915977 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:07.916170 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:07.916810 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:07.916815 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:07.917013 (+ 198us) client_negotiation.cc:769] Sending connection context
0810 06:38:07.917182 (+ 169us) client_negotiation.cc:240] Negotiation successful
0810 06:38:07.917516 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4435,"thread_start_us":4293,"threads_started":1}
I20260810 06:38:07.919275 10139 tablet_copy_service.cc:140] P 4f43ba0f328e4a03abc5b1f6d0226f78: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a63e2108523a4d96ac09190af21127a0 ({username='oryx', principal='oryx/127.31.59.253@KRBTEST.COM'} at 127.0.0.1:47266)
I20260810 06:38:07.919638 10139 tablet_copy_service.cc:161] P 4f43ba0f328e4a03abc5b1f6d0226f78: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a63e2108523a4d96ac09190af21127a0 at {username='oryx', principal='oryx/127.31.59.253@KRBTEST.COM'} at 127.0.0.1:47266: session id = a63e2108523a4d96ac09190af21127a0-00000000000000000000000000000000
I20260810 06:38:07.924714 10139 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 06:38:07.927825 10902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 06:38:07.934986 10902 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: tablet copy: Starting download of 0 data blocks...
I20260810 06:38:07.935443 10902 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: tablet copy: Starting download of 1 WAL segments...
I20260810 06:38:07.940164 10902 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 06:38:07.945508 10139 tablet_copy_service.cc:342] P 4f43ba0f328e4a03abc5b1f6d0226f78: Request end of tablet copy session a63e2108523a4d96ac09190af21127a0-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.59.253@KRBTEST.COM'} at 127.0.0.1:47266
I20260810 06:38:07.945799 10139 tablet_copy_service.cc:433] P 4f43ba0f328e4a03abc5b1f6d0226f78: ending tablet copy session a63e2108523a4d96ac09190af21127a0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a63e2108523a4d96ac09190af21127a0
I20260810 06:38:07.959270 10902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:07.962095 11079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:07.966724 11080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:07.968254 11082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:07.971560 10902 server_base.cc:1034] running on GCE node
I20260810 06:38:07.971886 10902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:07.972668 10902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:07.973893 10902 hybrid_clock.cc:648] HybridClock initialized: now 1786343887973821 us; error 36 us; skew 500 ppm
I20260810 06:38:07.974124 10902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:07 dist-test-slave-4tdj krb5kdc[10030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343887, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:07.976346 10902 init.cc:377] Logged in from keytab as oryx/127.31.59.253@KRBTEST.COM (short username oryx)
I20260810 06:38:07.977743 10902 webserver.cc:460] Webserver started at http://127.31.59.253:41297/ using document root <none> and password file <none>
I20260810 06:38:07.978344 10902 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:07.978439 10902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:07.981580 10902 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260810 06:38:07.983558 11093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:07.984020 10902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:07.984170 10902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/wal
uuid: "a63e2108523a4d96ac09190af21127a0"
format_stamp: "Formatted at 2026-08-10 06:38:07 on dist-test-slave-4tdj"
server_key: "a7a868195adefd3b8ad503de0db1a719"
server_key_iv: "b2313673f0169eb3bdb903233fd6ce96"
server_key_version: "encryptionkey@0"
I20260810 06:38:07.984342 10902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:07.998540 10902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:08.002552 10902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:08.037086 10902 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.253:37507
I20260810 06:38:08.037130 11176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.253:37507 every 8 connection(s)
I20260810 06:38:08.038246 10902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-1/data/info.pb
I20260810 06:38:08.043776 11177 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:38:08.047940 11177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: Bootstrap starting.
I20260810 06:38:08.068629 11177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:38:08.069592 11177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: Bootstrap complete.
I20260810 06:38:08.070884 11177 raft_consensus.cc:348] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: NON_VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: true } }
I20260810 06:38:08.071251 11177 raft_consensus.cc:729] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a63e2108523a4d96ac09190af21127a0, State: Initialized, Role: LEARNER
I20260810 06:38:08.071784 11177 consensus_queue.cc:260] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: NON_VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: true } }
I20260810 06:38:08.072899 11185 sys_catalog.cc:455] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: NON_VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: true } } }
I20260810 06:38:08.073364 11185 sys_catalog.cc:458] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [sys.catalog]: This master's current role is: LEARNER
I20260810 06:38:08.073411 11177 sys_catalog.cc:564] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:08.086674 11205 catalog_manager.cc:1259] Loaded cluster ID: 1a7d2e9180204d73b144bbdde26e2e50
I20260810 06:38:08.086797 11205 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: loading cluster ID for follower catalog manager: success
I20260810 06:38:08.090461 11205 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: acquiring CA information for follower catalog manager: success
I20260810 06:38:08.093674 11205 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:38:08.178381 10894 heartbeater.cc:503] Master 127.31.59.254:36175 was elected leader, sending a full tablet report...
W20260810 06:38:08.264731 11060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:08.265050 11060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:08.265195 11060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:08.273712 11060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:08.273829 11060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:08.273905 11060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:08.274068 11060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:08.274215 11060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:08.285398 11060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:38421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:36175,127.31.59.253:37507,127.31.59.252:37537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.252.keytab
--principal=oryx/127.31.59.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.252:37537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.59.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:08.286761 11060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:08.288028 11060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:08.301626 11214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:08.301785 11216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:08.301882 11219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:08.303315 11060 server_base.cc:1034] running on GCE node
I20260810 06:38:08.304237 11060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:08.306164 11060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:08.307456 11060 hybrid_clock.cc:648] HybridClock initialized: now 1786343888307373 us; error 65 us; skew 500 ppm
I20260810 06:38:08.307812 11060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:08 dist-test-slave-4tdj krb5kdc[10030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343888, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:08.311396 11060 init.cc:377] Logged in from keytab as oryx/127.31.59.252@KRBTEST.COM (short username oryx)
I20260810 06:38:08.312959 11060 webserver.cc:460] Webserver started at http://127.31.59.252:46385/ using document root <none> and password file <none>
I20260810 06:38:08.313738 11060 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:08.313862 11060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:08.314159 11060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:08.314278 11225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.307120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43722 (local address 127.31.59.253:37507)
0810 06:38:08.307747 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:08.307752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:08.307768 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:38:08.307828 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:08.307831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:08.307912 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:08.308023 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:08.308425 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.309849 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.311867 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.313201 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.313368 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.313627 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:08.313793 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 06:38:08.313943 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":293,"threads_started":1}
I20260810 06:38:08.314324 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.306909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:37507 (local address 127.0.0.1:43722)
0810 06:38:08.307092 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:08.307110 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:08.307217 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:08.308081 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:08.308087 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:08.308116 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:08.308281 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:08.308290 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.310006 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.310012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:08.311725 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:08.311739 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.313406 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.313412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:08.313608 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.313671 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 06:38:08.313756 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 06:38:08.313906 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 06:38:08.315604 11135 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260810 06:38:08.316207 11185 sys_catalog.cc:455] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f43ba0f328e4a03abc5b1f6d0226f78. Latest consensus state: current_term: 1 leader_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: NON_VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: true } } }
I20260810 06:38:08.316457 11185 sys_catalog.cc:458] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [sys.catalog]: This master's current role is: LEARNER
I20260810 06:38:08.318106 11060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/data/instance:
uuid: "5b6624b357734638a9272097b85844c9"
format_stamp: "Formatted at 2026-08-10 06:38:08 on dist-test-slave-4tdj"
server_key: "2e0818e4af0dd7caefde98bc4b9e63c2"
server_key_iv: "468a753e5ce32e35849b045f3b468797"
server_key_version: "encryptionkey@0"
I20260810 06:38:08.319129 11060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/wal/instance:
uuid: "5b6624b357734638a9272097b85844c9"
format_stamp: "Formatted at 2026-08-10 06:38:08 on dist-test-slave-4tdj"
server_key: "2e0818e4af0dd7caefde98bc4b9e63c2"
server_key_iv: "468a753e5ce32e35849b045f3b468797"
server_key_version: "encryptionkey@0"
I20260810 06:38:08.324474 11060 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 06:38:08.328536 11232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:08.329942 11060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:38:08.330113 11060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/wal
uuid: "5b6624b357734638a9272097b85844c9"
format_stamp: "Formatted at 2026-08-10 06:38:08 on dist-test-slave-4tdj"
server_key: "2e0818e4af0dd7caefde98bc4b9e63c2"
server_key_iv: "468a753e5ce32e35849b045f3b468797"
server_key_version: "encryptionkey@0"
I20260810 06:38:08.330286 11060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:08.357769 11060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:08.362211 11060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:08.362509 11060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:08.387537 11060 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.252:37537
I20260810 06:38:08.387744 11305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.252:37537 every 8 connection(s)
I20260810 06:38:08.388808 11060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/data/info.pb
I20260810 06:38:08.392663 11307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:08.397060 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 11060
I20260810 06:38:08.399590 11307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:38:08 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343888, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.252@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM
Aug 10 06:38:08 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.252@KRBTEST.COM
I20260810 06:38:08.419202 11022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.402123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47274 (local address 127.31.59.254:36175)
0810 06:38:08.402338 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:08.402344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:08.403219 (+ 875us) server_negotiation.cc:407] Connection header received
0810 06:38:08.403389 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:08.403395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:08.403453 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:08.403583 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:08.405299 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.405982 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.407711 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.408248 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.408413 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.412346 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:08.412351 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:08.412354 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:08.412391 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:08.415528 (+ 3137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.416371 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.416377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.416380 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.416580 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.417053 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.417059 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.417062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.417389 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:08.417546 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:08.418229 (+ 683us) server_negotiation.cc:299] Negotiation successful
0810 06:38:08.418575 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:38:08.419977 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.402003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.0.0.1:47274)
0810 06:38:08.403072 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:08.403127 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:08.403276 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:08.403650 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:08.403656 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:08.404504 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:08.405071 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:08.405090 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.406119 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.406123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:08.407554 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:08.407569 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.408553 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.408560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:08.408684 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.409243 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:08.409265 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:08.412150 (+ 2885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:08.415674 (+ 3524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:08.415686 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:08.415698 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:08.416236 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:08.416700 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:08.416705 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:08.416710 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:08.416920 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:08.417526 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:08.417534 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:08.417827 (+ 293us) client_negotiation.cc:769] Sending connection context
0810 06:38:08.418099 (+ 272us) client_negotiation.cc:240] Negotiation successful
0810 06:38:08.418624 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":839,"thread_start_us":684,"threads_started":1}
I20260810 06:38:08.419974 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.399202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49194 (local address 127.31.59.252:37537)
0810 06:38:08.399945 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:08.399956 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:08.399998 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:38:08.401717 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:08.401741 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:08.402195 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:08.402637 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:08.404260 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.405919 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.407287 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.407989 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.408122 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.413029 (+ 4907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:08.413038 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:08.413050 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:08.413079 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:08.416137 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.416770 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.416776 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.416778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.416976 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.417342 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.417346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.417348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.417814 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:08.418090 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:08.418429 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 06:38:08.418724 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":543,"thread_start_us":344,"threads_started":1}
I20260810 06:38:08.421624 11307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:38:08 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343888, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.252@KRBTEST.COM for oryx/127.31.59.253@KRBTEST.COM
I20260810 06:38:08.436113 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.422520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:37507 (local address 127.0.0.1:43732)
0810 06:38:08.422803 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:08.422818 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:08.422901 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:08.423154 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:08.423157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:08.423438 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:08.423560 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:08.423567 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.424572 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.424576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:08.425537 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:08.425548 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.426412 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.426418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:08.426574 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.427167 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:08.427201 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:08.429307 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:08.433556 (+ 4249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:08.433560 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:08.433563 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:08.433999 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:08.434574 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:08.434578 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:08.434580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:08.434714 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:08.435422 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:08.435427 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:08.435569 (+ 142us) client_negotiation.cc:769] Sending connection context
0810 06:38:08.435679 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 06:38:08.435879 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260810 06:38:08.436312 11225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.422385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43732 (local address 127.31.59.253:37507)
0810 06:38:08.422561 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:08.422567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:08.422883 (+ 316us) server_negotiation.cc:407] Connection header received
0810 06:38:08.422986 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:08.422990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:08.423056 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:08.423135 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:08.423665 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.424442 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.425642 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.426228 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.426349 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.429509 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:08.429514 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:08.429518 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:08.429570 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:08.433424 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.434136 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.434140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.434143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.434477 (+ 334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.434798 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.434802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.434804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.435287 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:08.435408 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:08.435771 (+ 363us) server_negotiation.cc:299] Negotiation successful
0810 06:38:08.436005 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:38:08.437386 11307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:38:08 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343888, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.252@KRBTEST.COM for oryx/127.31.59.252@KRBTEST.COM
I20260810 06:38:08.451191 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.438345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:37537 (local address 127.0.0.1:49202)
0810 06:38:08.438488 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:08.438508 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:08.438596 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:08.438914 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:08.438917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:08.439229 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:08.439363 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:08.439370 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.440557 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.440561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:08.441357 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:08.441367 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.442417 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.442421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:08.442632 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.443081 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:08.443096 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:08.445270 (+ 2174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:08.448526 (+ 3256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:08.448529 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:08.448531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:08.449046 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:08.449369 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:08.449372 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:08.449374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:08.449670 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:08.450194 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:08.450197 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:08.450461 (+ 264us) client_negotiation.cc:769] Sending connection context
0810 06:38:08.450628 (+ 167us) client_negotiation.cc:240] Negotiation successful
0810 06:38:08.450806 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":17}
I20260810 06:38:08.451306 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.438570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49202 (local address 127.31.59.252:37537)
0810 06:38:08.438705 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:08.438709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:08.438725 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:38:08.438763 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:08.438767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:08.438817 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:08.438896 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:08.439480 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.440430 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.441499 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.442238 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.442412 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.445515 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:08.445521 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:08.445525 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:08.445554 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:08.448382 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.449143 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.449147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.449149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.449273 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.449765 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.449768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.449770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.450077 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:08.450196 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:08.450728 (+ 532us) server_negotiation.cc:299] Negotiation successful
0810 06:38:08.450959 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":46}
I20260810 06:38:08.456877 11307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: Bootstrap starting.
I20260810 06:38:08.460724 11307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:08.462119 11307 log.cc:824] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:08.465395 11307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: No bootstrap required, opened a new log
I20260810 06:38:08.474207 11307 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } }
I20260810 06:38:08.474594 11307 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:08.474710 11307 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b6624b357734638a9272097b85844c9, State: Initialized, Role: FOLLOWER
I20260810 06:38:08.475247 11307 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } }
I20260810 06:38:08.476404 11315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } } }
I20260810 06:38:08.476999 11315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:38:08.477208 11307 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 06:38:08.490329 11331 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:38:08.490514 11331 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:38:08.493579 11060 master_runner.cc:428] Detected that this master 5b6624b357734638a9272097b85844c9 is joining an existing cluster
I20260810 06:38:08.493731 11060 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.59.252:37537
I20260810 06:38:08.527565 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 10202
I20260810 06:38:08.528698 11225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.499721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43748 (local address 127.31.59.253:37507)
0810 06:38:08.499922 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:08.499928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:08.505821 (+ 5893us) server_negotiation.cc:407] Connection header received
0810 06:38:08.506526 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:08.506535 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:08.506624 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:08.506708 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:08.510151 (+ 3443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.511266 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.515676 (+ 4410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.516738 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.516859 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.520887 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:08.520894 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:08.520899 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:08.520939 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:08.524732 (+ 3793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.525965 (+ 1233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.525971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.525974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.526135 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.526630 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.526635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.526638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.526989 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:08.527134 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:08.527866 (+ 732us) server_negotiation.cc:299] Negotiation successful
0810 06:38:08.528149 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:38:08.532935 11348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.500714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:37507 (local address 127.0.0.1:43748)
0810 06:38:08.505497 (+ 4783us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:08.505520 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:08.506315 (+ 795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:08.507022 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:08.507028 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:08.508442 (+ 1414us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:08.509951 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:08.509968 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.511845 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.511851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:08.515514 (+ 3663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:08.515527 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.517486 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.517495 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:08.517867 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.518886 (+ 1019us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:08.518910 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:08.520646 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:08.525328 (+ 4682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:08.525333 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:08.525337 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:08.525822 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:08.526266 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:08.526270 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:08.526274 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:08.526501 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:08.527216 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:08.527221 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:08.527523 (+ 302us) client_negotiation.cc:769] Sending connection context
0810 06:38:08.529225 (+ 1702us) client_negotiation.cc:240] Negotiation successful
0810 06:38:08.532004 (+ 2779us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4599,"thread_start_us":7981,"threads_started":1}
I20260810 06:38:08.536384 11022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.499421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47276 (local address 127.31.59.254:36175)
0810 06:38:08.499737 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:08.499744 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:08.509942 (+ 10198us) server_negotiation.cc:407] Connection header received
0810 06:38:08.510209 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:08.510216 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:08.510327 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:08.510407 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:08.514974 (+ 4567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.516400 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.519157 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.519831 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.519936 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.523350 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:08.523357 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:08.523362 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:08.523404 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:08.531095 (+ 7691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.532905 (+ 1810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.532911 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.532914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.533156 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.533811 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.533817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.533820 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.534512 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:08.534642 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:08.535625 (+ 983us) server_negotiation.cc:299] Negotiation successful
0810 06:38:08.535852 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260810 06:38:08.537390 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.499696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49218 (local address 127.31.59.252:37537)
0810 06:38:08.500111 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:08.500116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:08.515029 (+ 14913us) server_negotiation.cc:407] Connection header received
0810 06:38:08.515257 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:08.515264 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:08.515346 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:08.515428 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:08.518005 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.519486 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.522699 (+ 3213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.523498 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.523589 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.526178 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:08.526185 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:08.526189 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:08.526225 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:08.532240 (+ 6015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.534184 (+ 1944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.534189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.534192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.534624 (+ 432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.535089 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.535094 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.535098 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.535665 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:08.535867 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:08.536442 (+ 575us) server_negotiation.cc:299] Negotiation successful
0810 06:38:08.536749 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254}
I20260810 06:38:08.538393 11347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.500759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.0.0.1:47276)
0810 06:38:08.509674 (+ 8915us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:08.509699 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:08.510062 (+ 363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:08.510683 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:08.510689 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:08.513182 (+ 2493us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:08.514767 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:08.514783 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.516594 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.516599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:08.518976 (+ 2377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:08.518992 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.520310 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.520318 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:08.520526 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.521201 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:08.521223 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:08.523128 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:08.531654 (+ 8526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:08.531659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:08.531662 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:08.532750 (+ 1088us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:08.533387 (+ 637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:08.533391 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:08.533395 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:08.533672 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:08.534725 (+ 1053us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:08.534731 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:08.535196 (+ 465us) client_negotiation.cc:769] Sending connection context
0810 06:38:08.536891 (+ 1695us) client_negotiation.cc:240] Negotiation successful
0810 06:38:08.537379 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8735,"mutex_wait_us":3,"thread_start_us":5093,"threads_started":1}
I20260810 06:38:08.538727 11349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.500822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:37537 (local address 127.0.0.1:49218)
0810 06:38:08.514747 (+ 13925us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:08.514767 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:08.515125 (+ 358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:08.515624 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:08.515629 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:08.516470 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:08.517850 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:08.517864 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.519840 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.519845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:08.522528 (+ 2683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:08.522544 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.523922 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.523929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:08.524086 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.524725 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:08.524745 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:08.526027 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:08.533453 (+ 7426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:08.533458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:08.533462 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:08.534057 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:08.534758 (+ 701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:08.534762 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:08.534765 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:08.534963 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:08.535956 (+ 993us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:08.535959 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:08.536184 (+ 225us) client_negotiation.cc:769] Sending connection context
0810 06:38:08.537573 (+ 1389us) client_negotiation.cc:240] Negotiation successful
0810 06:38:08.537957 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13779,"mutex_wait_us":19,"thread_start_us":12606,"threads_started":1}
Aug 10 06:38:08 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.193@KRBTEST.COM
I20260810 06:38:08.553315 11350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.529045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42918 (local address 127.31.59.193:33401)
0810 06:38:08.533539 (+ 4494us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:08.533548 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:08.533576 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:38:08.533644 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:08.533653 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:08.533949 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:08.536029 (+ 2080us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:08.536603 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.538923 (+ 2320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.540790 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.541796 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.545087 (+ 3291us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.547078 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:08.547089 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:08.547103 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:08.547138 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:08.550082 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.550822 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.550832 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.550836 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.550970 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.551392 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.551395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.551397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.551712 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:08.552098 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:08.552360 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 06:38:08.552669 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4322,"thread_start_us":4059,"threads_started":1}
Aug 10 06:38:08 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343885, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.254@KRBTEST.COM for oryx/127.31.59.252@KRBTEST.COM
I20260810 06:38:08.555686 10427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:08.562872 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.545756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49230 (local address 127.31.59.252:37537)
0810 06:38:08.546143 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:08.546148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:08.546166 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:38:08.546406 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:08.546412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:08.546510 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:08.546611 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:08.547658 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.548777 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.550565 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.551496 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.552273 (+ 777us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.555398 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:08.555406 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:08.555411 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:08.555449 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:08.559652 (+ 4203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.560488 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.560494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.560497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.560648 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.561163 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.561169 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.561173 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.561522 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:08.561679 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:08.562266 (+ 587us) server_negotiation.cc:299] Negotiation successful
0810 06:38:08.562519 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263}
I20260810 06:38:08.562884 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.545754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:37537 (local address 127.0.0.1:49230)
0810 06:38:08.546091 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:08.546112 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:08.546220 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:08.546654 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:08.546658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:08.547143 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:08.547475 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:08.547491 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.548935 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.548940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:08.550364 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:08.550379 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.551702 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.551708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:08.551880 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.552528 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:08.552564 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:08.555135 (+ 2571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:08.559793 (+ 4658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:08.559799 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:08.559803 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:08.560345 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:08.560761 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:08.560767 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:08.560771 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:08.561019 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:08.561650 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:08.561655 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:08.561966 (+ 311us) client_negotiation.cc:769] Sending connection context
0810 06:38:08.562120 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 06:38:08.562458 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260810 06:38:08.564512 10107 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.59.252:37537: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5b6624b357734638a9272097b85844c9"
member_type: NON_VOTER
last_known_addr {
host: "127.31.59.252"
port: 37537
}
attrs {
promote: true
}
}
dest_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78"
cas_config_opid_index: 5
I20260810 06:38:08.565764 10107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: NON_VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: true } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: NON_VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: true } }
I20260810 06:38:08.568092 11056 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5b6624b357734638a9272097b85844c9 (127.31.59.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: NON_VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: true } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: NON_VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: true } } }
I20260810 06:38:08.572190 11056 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.59.252:37537
I20260810 06:38:08.572832 11060 master.cc:537] Master@127.31.59.252:37537 shutting down...
I20260810 06:38:08.574244 11135 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 1 LEARNER]: Refusing update from remote peer 4f43ba0f328e4a03abc5b1f6d0226f78: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20260810 06:38:08.575604 10075 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 -> Peer 5b6624b357734638a9272097b85844c9 (127.31.59.252:37537): Couldn't send request to peer 5b6624b357734638a9272097b85844c9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 06:38:08.586632 11352 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: NON_VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 06:38:08.589214 11135 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5b6624b357734638a9272097b85844c9 (127.31.59.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: NON_VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: true } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: NON_VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: true } } }
I20260810 06:38:08.591195 11354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: NON_VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: true } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: NON_VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: true } } }
I20260810 06:38:08.591554 11354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:08.593715 11185 sys_catalog.cc:455] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: NON_VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: true } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: NON_VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: true } } }
I20260810 06:38:08.594007 11185 sys_catalog.cc:458] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [sys.catalog]: This master's current role is: LEARNER
I20260810 06:38:08.595126 11352 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78: attempting to promote NON_VOTER a63e2108523a4d96ac09190af21127a0 to VOTER
I20260810 06:38:08.595854 11352 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: NON_VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: true } }
I20260810 06:38:08.597922 11135 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 1 LEARNER]: Refusing update from remote peer 4f43ba0f328e4a03abc5b1f6d0226f78: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 06:38:08.598620 11056 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260810 06:38:08.598834 10075 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 -> Peer 5b6624b357734638a9272097b85844c9 (127.31.59.252:37537): Couldn't send request to peer 5b6624b357734638a9272097b85844c9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 06:38:08.603786 11358 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a63e2108523a4d96ac09190af21127a0 (127.31.59.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: NON_VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: true } } }
I20260810 06:38:08.604290 11135 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a63e2108523a4d96ac09190af21127a0 (127.31.59.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: NON_VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: true } } }
I20260810 06:38:08.608487 11185 sys_catalog.cc:455] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: NON_VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: true } } }
I20260810 06:38:08.608726 11185 sys_catalog.cc:458] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:38:08.609093 11056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: NON_VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: true } } }
I20260810 06:38:08.609324 11056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:08.609627 11056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: NON_VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: true } } }
I20260810 06:38:08.609859 11056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:08.612932 11060 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:08.613351 11060 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:08.613574 11060 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: stopping tablet replica
I20260810 06:38:08.620862 11060 master.cc:552] Master@127.31.59.252:37537 shutdown complete.
I20260810 06:38:08.620990 11060 master_runner.cc:305] Clearing existing system tablet
I20260810 06:38:08.621428 11060 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:08.621595 11060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:08.630720 11060 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.006s sys 0.002s
I20260810 06:38:08.632417 11365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:08.632711 11060 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:08.632983 11060 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260810 06:38:08.633082 11060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/wal
uuid: "5b6624b357734638a9272097b85844c9"
format_stamp: "Formatted at 2026-08-10 06:38:08 on dist-test-slave-4tdj"
server_key: "2e0818e4af0dd7caefde98bc4b9e63c2"
server_key_iv: "468a753e5ce32e35849b045f3b468797"
server_key_version: "encryptionkey@0"
I20260810 06:38:08.636431 11060 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:08.641531 11060 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 06:38:08.641731 11060 log.cc:1196] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 06:38:08.642194 11060 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: Deleting consensus metadata
I20260810 06:38:08.642431 11060 master_runner.cc:315] Copying system tablet from 127.31.59.254:36175
I20260810 06:38:08.645308 11060 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: tablet copy: Beginning tablet copy session from remote peer at address 127.31.59.254:36175
I20260810 06:38:08.659572 11022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.646189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47292 (local address 127.31.59.254:36175)
0810 06:38:08.646810 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:08.646818 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:08.647695 (+ 877us) server_negotiation.cc:407] Connection header received
0810 06:38:08.647902 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:08.647908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:08.647987 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:08.648069 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:08.648989 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.649797 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.651193 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.651591 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.651703 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.653504 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:08.653508 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:08.653512 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:08.653543 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:08.656658 (+ 3115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.657341 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.657346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.657350 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.657492 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.657837 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.657842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.657846 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.658257 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:08.658418 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:08.659025 (+ 607us) server_negotiation.cc:299] Negotiation successful
0810 06:38:08.659269 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427}
I20260810 06:38:08.660409 11372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.646070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.0.0.1:47292)
0810 06:38:08.647576 (+ 1506us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:08.647601 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:08.647775 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:08.648117 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:08.648122 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:08.648642 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:08.648835 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:08.648844 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.649938 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.649944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:08.651059 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:08.651073 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.651740 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.651746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:08.651913 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.652499 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:08.652516 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:08.653304 (+ 788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:08.656823 (+ 3519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:08.656828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:08.656831 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:08.657228 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:08.657597 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:08.657600 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:08.657603 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:08.657736 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:08.658496 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:08.658501 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:08.658713 (+ 212us) client_negotiation.cc:769] Sending connection context
0810 06:38:08.659807 (+ 1094us) client_negotiation.cc:240] Negotiation successful
0810 06:38:08.660067 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1356,"thread_start_us":1200,"threads_started":1}
I20260810 06:38:08.661127 10139 tablet_copy_service.cc:140] P 4f43ba0f328e4a03abc5b1f6d0226f78: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5b6624b357734638a9272097b85844c9 ({username='oryx', principal='oryx/127.31.59.252@KRBTEST.COM'} at 127.0.0.1:47292)
I20260810 06:38:08.661283 10139 tablet_copy_service.cc:161] P 4f43ba0f328e4a03abc5b1f6d0226f78: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5b6624b357734638a9272097b85844c9 at {username='oryx', principal='oryx/127.31.59.252@KRBTEST.COM'} at 127.0.0.1:47292: session id = 5b6624b357734638a9272097b85844c9-00000000000000000000000000000000
I20260810 06:38:08.663879 10139 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 06:38:08.665464 11060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 06:38:08.671151 11060 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: tablet copy: Starting download of 0 data blocks...
I20260810 06:38:08.671398 11060 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: tablet copy: Starting download of 1 WAL segments...
I20260810 06:38:08.673410 11060 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 06:38:08.677008 10139 tablet_copy_service.cc:342] P 4f43ba0f328e4a03abc5b1f6d0226f78: Request end of tablet copy session 5b6624b357734638a9272097b85844c9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.59.252@KRBTEST.COM'} at 127.0.0.1:47292
I20260810 06:38:08.677179 10139 tablet_copy_service.cc:433] P 4f43ba0f328e4a03abc5b1f6d0226f78: ending tablet copy session 5b6624b357734638a9272097b85844c9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5b6624b357734638a9272097b85844c9
I20260810 06:38:08.695528 11060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:38:08.698244 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 10202
W20260810 06:38:08.701787 11373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:08.706854 11374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:08.726641 11060 server_base.cc:1034] running on GCE node
W20260810 06:38:08.727713 11378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:08.728251 11060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:08.729046 11060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:08.730265 11060 hybrid_clock.cc:648] HybridClock initialized: now 1786343888730207 us; error 40 us; skew 500 ppm
I20260810 06:38:08.730578 11060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:08 dist-test-slave-4tdj krb5kdc[10030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343888, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:08.732916 11060 init.cc:377] Logged in from keytab as oryx/127.31.59.252@KRBTEST.COM (short username oryx)
I20260810 06:38:08.739799 11060 webserver.cc:460] Webserver started at http://127.31.59.252:43723/ using document root <none> and password file <none>
I20260810 06:38:08.740196 11060 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:08.740327 11060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:08.741794 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:33401
--local_ip_for_outbound_sockets=127.31.59.193
--tserver_master_addrs=127.31.59.254:36175
--webserver_port=45543
--webserver_interface=127.31.59.193
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.193.keytab
--principal=oryx/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:38421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.59.254:36175,127.31.59.253:37507
--tserver_master_addrs=127.31.59.254:36175,127.31.59.253:37507,127.31.59.252:37537 with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:38:08.747046 11060 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.000s sys 0.007s
I20260810 06:38:08.750273 11395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:08.750857 11060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 06:38:08.751034 11060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/wal
uuid: "5b6624b357734638a9272097b85844c9"
format_stamp: "Formatted at 2026-08-10 06:38:08 on dist-test-slave-4tdj"
server_key: "2e0818e4af0dd7caefde98bc4b9e63c2"
server_key_iv: "468a753e5ce32e35849b045f3b468797"
server_key_version: "encryptionkey@0"
I20260810 06:38:08.751335 11060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:08.764947 11060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:08.769330 11060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:08.804489 11060 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.252:37537
I20260810 06:38:08.804579 11480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.252:37537 every 8 connection(s)
I20260810 06:38:08.805490 11060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/master-2/data/info.pb
I20260810 06:38:08.811549 11481 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:38:08.815694 11481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: Bootstrap starting.
I20260810 06:38:08.836753 11481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:38:08.837378 11481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: Bootstrap complete.
I20260810 06:38:08.838698 11481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: NON_VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: true } }
I20260810 06:38:08.839082 11481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b6624b357734638a9272097b85844c9, State: Initialized, Role: LEARNER
I20260810 06:38:08.839434 11481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: NON_VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: true } }
I20260810 06:38:08.840329 11485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: NON_VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: true } } }
I20260810 06:38:08.840791 11481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:08.840811 11485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [sys.catalog]: This master's current role is: LEARNER
I20260810 06:38:08.851712 11502 catalog_manager.cc:1259] Loaded cluster ID: 1a7d2e9180204d73b144bbdde26e2e50
I20260810 06:38:08.851814 11502 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: loading cluster ID for follower catalog manager: success
I20260810 06:38:08.855212 11502 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: acquiring CA information for follower catalog manager: success
I20260810 06:38:08.857447 11502 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:38:09.086750 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.079334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49236 (local address 127.31.59.252:37537)
0810 06:38:09.080474 (+ 1140us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:09.080479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:09.080497 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:38:09.080567 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:09.080571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:09.080648 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:09.080761 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:09.081261 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.082396 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.084641 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.085815 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.085921 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.086085 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:09.086403 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 06:38:09.086510 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1020,"thread_start_us":876,"threads_started":1}
I20260810 06:38:09.086831 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.079135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:37537 (local address 127.0.0.1:49236)
0810 06:38:09.080129 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:09.080159 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:09.080369 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:09.080811 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:09.080817 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:09.080860 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:09.081096 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:09.081119 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.082547 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.082553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:09.084483 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:09.084499 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.086002 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.086009 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:09.086210 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.086248 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 06:38:09.086371 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 06:38:09.086506 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"thread_start_us":687,"threads_started":1}
I20260810 06:38:09.088145 11441 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 06:38:09.088891 11485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f43ba0f328e4a03abc5b1f6d0226f78. Latest consensus state: current_term: 1 leader_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: NON_VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: true } } }
I20260810 06:38:09.089210 11485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [sys.catalog]: This master's current role is: LEARNER
W20260810 06:38:09.125910 11397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:09.126240 11397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:09.126458 11397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:09.134699 11397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:09.134811 11397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:09.134966 11397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:38:09.145609 11397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:38421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.193.keytab
--principal=oryx/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:33401
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=45543
--tserver_master_addrs=127.31.59.254:36175,127.31.59.253:37507,127.31.59.252:37537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:09.146916 11397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:09.148224 11397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:09.162633 11520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:09.162660 11518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:09.163210 11517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:09.163615 11397 server_base.cc:1034] running on GCE node
I20260810 06:38:09.164216 11397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:09.165099 11397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:09.166329 11397 hybrid_clock.cc:648] HybridClock initialized: now 1786343889166232 us; error 44 us; skew 500 ppm
I20260810 06:38:09.166637 11397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:09 dist-test-slave-4tdj krb5kdc[10030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343889, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:09.169812 11397 init.cc:377] Logged in from keytab as oryx/127.31.59.193@KRBTEST.COM (short username oryx)
I20260810 06:38:09.171169 11397 webserver.cc:460] Webserver started at http://127.31.59.193:45543/ using document root <none> and password file <none>
I20260810 06:38:09.171857 11397 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:09.172006 11397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:09.177858 11397 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 06:38:09.180933 11533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:09.182448 11397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:38:09.182579 11397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "7862319cb6c14028878f112d67842e80"
format_stamp: "Formatted at 2026-08-10 06:38:06 on dist-test-slave-4tdj"
server_key: "8485d1d4f0cb8a3d4ef370e94b462d81"
server_key_iv: "a354a3e0ad76676618f57ccc1ad9930d"
server_key_version: "encryptionkey@0"
I20260810 06:38:09.183758 11397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:09.207252 11397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:09.211445 11397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:09.211702 11397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:09.212600 11397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:09.214092 11397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:09.214181 11397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:09.214258 11397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:09.214332 11397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:09.256385 11397 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:33401
I20260810 06:38:09.256522 11729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:33401 every 8 connection(s)
I20260810 06:38:09.257782 11397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:38:09.261245 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 11397
I20260810 06:38:09.261600 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 10444
Aug 10 06:38:09 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343889, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM
Aug 10 06:38:09 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343889, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.252@KRBTEST.COM
Aug 10 06:38:09 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.194@KRBTEST.COM
Aug 10 06:38:09 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343889, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.253@KRBTEST.COM
I20260810 06:38:09.298338 11736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.260656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:55109 (local address 127.31.59.254:36175)
0810 06:38:09.272967 (+ 12311us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:09.272974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:09.277531 (+ 4557us) server_negotiation.cc:407] Connection header received
0810 06:38:09.277680 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:09.277686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:09.277764 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:09.277880 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:09.281293 (+ 3413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.282157 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.285175 (+ 3018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.285667 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.285777 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.290822 (+ 5045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:09.290827 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:09.290831 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:09.290863 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:09.294149 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.295234 (+ 1085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.295240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.295243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.295457 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.296132 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.296138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.296141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.296485 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:09.296647 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:09.297683 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0810 06:38:09.297915 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12168,"thread_start_us":12304,"threads_started":1}
I20260810 06:38:09.298514 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.262282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:53747 (local address 127.31.59.252:37537)
0810 06:38:09.263140 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:09.263147 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:09.278379 (+ 15232us) server_negotiation.cc:407] Connection header received
0810 06:38:09.279394 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:09.279400 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:09.279495 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:09.279587 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:09.282965 (+ 3378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.284050 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.286339 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.287032 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.287129 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.292080 (+ 4951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:09.292086 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:09.292091 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:09.292134 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:09.295568 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.296295 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.296301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.296305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.296429 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.296911 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.296916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.296919 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.297276 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:09.297401 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:09.297922 (+ 521us) server_negotiation.cc:299] Negotiation successful
0810 06:38:09.298152 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 06:38:09.300794 11743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.264624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.193:55109)
0810 06:38:09.277416 (+ 12792us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:09.277437 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:09.277573 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:09.279352 (+ 1779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:09.279356 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:09.280746 (+ 1390us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:09.281094 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:09.281107 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.282461 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.282466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:09.285042 (+ 2576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:09.285057 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.286004 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.286014 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:09.286349 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.287083 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:09.287121 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:09.290568 (+ 3447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:09.294334 (+ 3766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:09.294343 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:09.294360 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:09.295089 (+ 729us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:09.295715 (+ 626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:09.295721 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:09.295725 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:09.295993 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:09.296813 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:09.296821 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:09.297231 (+ 410us) client_negotiation.cc:769] Sending connection context
0810 06:38:09.298781 (+ 1550us) client_negotiation.cc:240] Negotiation successful
0810 06:38:09.299453 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12634,"mutex_wait_us":399,"thread_start_us":13167,"threads_started":1}
I20260810 06:38:09.300822 11741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.265040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:37537 (local address 127.31.59.193:53747)
0810 06:38:09.278233 (+ 13193us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:09.278252 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:09.279208 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:09.279886 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:09.279892 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:09.281998 (+ 2106us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:09.282756 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:09.282775 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.284227 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.284231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:09.286148 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:09.286163 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.287452 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.287460 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:09.287689 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.288305 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:09.288332 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:09.291855 (+ 3523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:09.295674 (+ 3819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:09.295679 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:09.295683 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:09.296150 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:09.296579 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:09.296589 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:09.296592 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:09.296779 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:09.297486 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:09.297491 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:09.297633 (+ 142us) client_negotiation.cc:769] Sending connection context
0810 06:38:09.298757 (+ 1124us) client_negotiation.cc:240] Negotiation successful
0810 06:38:09.299315 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13044,"mutex_wait_us":27,"thread_start_us":8837,"threads_started":1}
I20260810 06:38:09.302582 11730 heartbeater.cc:347] Connected to a master server at 127.31.59.252:37537
I20260810 06:38:09.302856 11730 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:09.303232 11739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.262981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35328 (local address 127.31.59.194:36093)
0810 06:38:09.280494 (+ 17513us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:09.280503 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:09.280531 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:38:09.280628 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:09.280637 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:09.280874 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:09.281053 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:09.283342 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.284805 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.286682 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.287498 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.287630 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.294476 (+ 6846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:09.294492 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:09.294511 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:09.294565 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:09.299502 (+ 4937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.300740 (+ 1238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.300749 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.300752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.300918 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.301363 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.301368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.301371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.301797 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:09.302077 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:09.302640 (+ 563us) server_negotiation.cc:299] Negotiation successful
0810 06:38:09.302886 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17345,"thread_start_us":17174,"threads_started":1}
I20260810 06:38:09.303753 11730 heartbeater.cc:511] Master 127.31.59.252:37537 requested a full tablet report, sending...
I20260810 06:38:09.304450 11731 heartbeater.cc:347] Connected to a master server at 127.31.59.254:36175
I20260810 06:38:09.304585 11731 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:09.305231 11731 heartbeater.cc:511] Master 127.31.59.254:36175 requested a full tablet report, sending...
I20260810 06:38:09.306178 11424 ts_manager.cc:194] Registered new tserver with Master: 7862319cb6c14028878f112d67842e80 (127.31.59.193:33401)
I20260810 06:38:09.306638 10107 ts_manager.cc:194] Re-registered known tserver with Master: 7862319cb6c14028878f112d67842e80 (127.31.59.193:33401)
I20260810 06:38:09.307304 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.260655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:50265 (local address 127.31.59.253:37507)
0810 06:38:09.273382 (+ 12727us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:09.273388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:09.274652 (+ 1264us) server_negotiation.cc:407] Connection header received
0810 06:38:09.276740 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:09.276746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:09.276851 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:09.276958 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:09.280198 (+ 3240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.281465 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.286079 (+ 4614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.287298 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.287397 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.294155 (+ 6758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:09.294162 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:09.294166 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:09.294206 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:09.298269 (+ 4063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.300266 (+ 1997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.300272 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.300276 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.300423 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.300894 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.300900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.300904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.301626 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:09.301758 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:09.306694 (+ 4936us) server_negotiation.cc:299] Negotiation successful
0810 06:38:09.306904 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12606,"thread_start_us":12512,"threads_started":1}
I20260810 06:38:09.307780 10107 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:55109
I20260810 06:38:09.308148 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.264117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:37507 (local address 127.31.59.193:50265)
0810 06:38:09.274438 (+ 10321us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:09.274529 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:09.276442 (+ 1913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:09.277422 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:09.277434 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:09.278897 (+ 1463us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:09.279980 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:09.280008 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.282461 (+ 2453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.282466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:09.285042 (+ 2576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:09.285057 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.287863 (+ 2806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.287872 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:09.288431 (+ 559us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.289517 (+ 1086us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:09.289543 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:09.293654 (+ 4111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:09.299673 (+ 6019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:09.299677 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:09.299681 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:09.300134 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:09.300563 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:09.300567 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:09.300570 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:09.300752 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:09.301839 (+ 1087us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:09.301843 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:09.302004 (+ 161us) client_negotiation.cc:769] Sending connection context
0810 06:38:09.307496 (+ 5492us) client_negotiation.cc:240] Negotiation successful
0810 06:38:09.307769 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9931,"mutex_wait_us":119,"thread_start_us":12777,"threads_started":1}
I20260810 06:38:09.308513 10647 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:09.308943 11733 heartbeater.cc:347] Connected to a master server at 127.31.59.253:37507
I20260810 06:38:09.309078 11733 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:09.309587 11733 heartbeater.cc:511] Master 127.31.59.253:37507 requested a full tablet report, sending...
I20260810 06:38:09.311916 11122 ts_manager.cc:194] Registered new tserver with Master: 7862319cb6c14028878f112d67842e80 (127.31.59.193:33401)
I20260810 06:38:09.392349 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 10444
I20260810 06:38:09.416215 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:36093
--local_ip_for_outbound_sockets=127.31.59.194
--tserver_master_addrs=127.31.59.254:36175
--webserver_port=45857
--webserver_interface=127.31.59.194
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.194.keytab
--principal=oryx/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:38421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.59.254:36175,127.31.59.253:37507
--tserver_master_addrs=127.31.59.254:36175,127.31.59.253:37507,127.31.59.252:37537 with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:38:09.602594 11358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: NON_VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: true } } }
I20260810 06:38:09.602972 11358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:09.604627 11750 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78: attempting to promote NON_VOTER 5b6624b357734638a9272097b85844c9 to VOTER
I20260810 06:38:09.605222 11750 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false } }
I20260810 06:38:09.607087 11135 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 1 FOLLOWER]: Refusing update from remote peer 4f43ba0f328e4a03abc5b1f6d0226f78: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 06:38:09.607307 11441 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [term 1 LEARNER]: Refusing update from remote peer 4f43ba0f328e4a03abc5b1f6d0226f78: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 06:38:09.607719 11752 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 06:38:09.608209 11358 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 06:38:09.621668 11358 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 5b6624b357734638a9272097b85844c9 (127.31.59.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false } } }
I20260810 06:38:09.626683 11750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false } } }
I20260810 06:38:09.626988 11750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:09.627252 11750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false } } }
I20260810 06:38:09.627480 11750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:09.627969 11133 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 5b6624b357734638a9272097b85844c9 (127.31.59.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false } } }
I20260810 06:38:09.631135 11760 sys_catalog.cc:455] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false } } }
I20260810 06:38:09.631606 11760 sys_catalog.cc:458] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:38:09.631758 11750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false } } }
I20260810 06:38:09.631976 11750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:09.632683 11441 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 5b6624b357734638a9272097b85844c9 (127.31.59.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false } } }
I20260810 06:38:09.637061 11764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false } } }
I20260810 06:38:09.637353 11764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 06:38:09.750236 11748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:09.750558 11748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:09.750705 11748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:09.758037 11748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:09.758134 11748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:09.758281 11748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:38:09.767732 11748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:38421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.194.keytab
--principal=oryx/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:36093
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=45857
--tserver_master_addrs=127.31.59.254:36175,127.31.59.253:37507,127.31.59.252:37537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:09.768913 11748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:09.770187 11748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:09.782888 11770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:09.782927 11771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:09.782958 11773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:09.784202 11748 server_base.cc:1034] running on GCE node
I20260810 06:38:09.784787 11748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:09.785681 11748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:09.786863 11748 hybrid_clock.cc:648] HybridClock initialized: now 1786343889786810 us; error 42 us; skew 500 ppm
I20260810 06:38:09.787163 11748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:09 dist-test-slave-4tdj krb5kdc[10030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343889, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:09.790161 11748 init.cc:377] Logged in from keytab as oryx/127.31.59.194@KRBTEST.COM (short username oryx)
I20260810 06:38:09.791451 11748 webserver.cc:460] Webserver started at http://127.31.59.194:45857/ using document root <none> and password file <none>
I20260810 06:38:09.792109 11748 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:09.792227 11748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:09.797278 11748 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260810 06:38:09.799857 11789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:09.801112 11748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:38:09.801245 11748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "3fa769dc3836400cb8b62f521a3aad4a"
format_stamp: "Formatted at 2026-08-10 06:38:06 on dist-test-slave-4tdj"
server_key: "315565a7ce08fd5b8690795f4188e08b"
server_key_iv: "fbbf2f45391d5e8e7256863f6e248ada"
server_key_version: "encryptionkey@0"
I20260810 06:38:09.802157 11748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:09.827497 11748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:09.831212 11748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:09.831465 11748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:09.832376 11748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:09.833730 11748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:09.833814 11748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:09.833891 11748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:09.833945 11748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:09.864133 11748 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:36093
I20260810 06:38:09.864199 11969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:36093 every 8 connection(s)
I20260810 06:38:09.865613 11748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:38:09.866570 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 11748
I20260810 06:38:09.866844 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 10664
Aug 10 06:38:09 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.195@KRBTEST.COM
Aug 10 06:38:09 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343889, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.252@KRBTEST.COM
Aug 10 06:38:09 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343889, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM
Aug 10 06:38:09 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343889, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.253@KRBTEST.COM
I20260810 06:38:09.909879 11975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.867854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58358 (local address 127.31.59.195:44247)
0810 06:38:09.879819 (+ 11965us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:09.879828 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:09.883215 (+ 3387us) server_negotiation.cc:407] Connection header received
0810 06:38:09.883303 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:09.883312 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:09.883585 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:09.883799 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:09.888209 (+ 4410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.890133 (+ 1924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.891908 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.892536 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.892639 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.900128 (+ 7489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:09.900146 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:09.900165 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:09.900240 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:09.905759 (+ 5519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.906679 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.906688 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.906692 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.906857 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.907304 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.907309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.907312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.907743 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:09.908043 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:09.909192 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0810 06:38:09.909460 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11783,"thread_start_us":11615,"threads_started":1}
I20260810 06:38:09.910887 11979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.881123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:47053 (local address 127.31.59.252:37537)
0810 06:38:09.882445 (+ 1322us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:09.882452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:09.882476 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:38:09.888833 (+ 6357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:09.888838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:09.888914 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:09.889009 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:09.893340 (+ 4331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.894587 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.897530 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.898261 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.898369 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.902949 (+ 4580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:09.902955 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:09.902960 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:09.902997 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:09.906543 (+ 3546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.907736 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.907742 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.907745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.908032 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.908606 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.908611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.908614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.909172 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:09.909292 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:09.910320 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0810 06:38:09.910538 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1152,"thread_start_us":1037,"threads_started":1}
I20260810 06:38:09.911213 11983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.884212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:58915 (local address 127.31.59.253:37507)
0810 06:38:09.887693 (+ 3481us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:09.887699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:09.888330 (+ 631us) server_negotiation.cc:407] Connection header received
0810 06:38:09.888831 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:09.888837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:09.888914 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:09.889021 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:09.893299 (+ 4278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.894447 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.897759 (+ 3312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.898400 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.898508 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.904043 (+ 5535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:09.904050 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:09.904056 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:09.904121 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:09.907749 (+ 3628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.908708 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.908713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.908716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.908856 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.909343 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.909348 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.909351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.909712 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:09.909849 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:09.910669 (+ 820us) server_negotiation.cc:299] Negotiation successful
0810 06:38:09.910873 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3359,"thread_start_us":3211,"threads_started":1}
I20260810 06:38:09.911557 10888 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:09.911890 11976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.868365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:54061 (local address 127.31.59.254:36175)
0810 06:38:09.887220 (+ 18855us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:09.887226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:09.887249 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:38:09.889404 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:09.889410 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:09.889537 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:09.889617 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:09.893324 (+ 3707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.894601 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.897519 (+ 2918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.898160 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.898256 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.903240 (+ 4984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:09.903246 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:09.903250 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:09.903299 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:09.907688 (+ 4389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.908594 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.908599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.908602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.908755 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.909235 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.909240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.909243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.909575 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:09.909757 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:09.911285 (+ 1528us) server_negotiation.cc:299] Negotiation successful
0810 06:38:09.911529 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18607,"thread_start_us":18707,"threads_started":1}
I20260810 06:38:09.912405 11981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.883735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:37507 (local address 127.31.59.194:58915)
0810 06:38:09.888236 (+ 4501us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:09.888251 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:09.888476 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:09.890452 (+ 1976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:09.890457 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:09.891105 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:09.893056 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:09.893112 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.894670 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.894676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:09.897593 (+ 2917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:09.897608 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.898894 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.898902 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:09.899151 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.900096 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:09.900120 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:09.903783 (+ 3663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:09.907901 (+ 4118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:09.907906 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:09.907910 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:09.908575 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:09.909000 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:09.909004 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:09.909008 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:09.909215 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:09.909948 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:09.909952 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:09.910413 (+ 461us) client_negotiation.cc:769] Sending connection context
0810 06:38:09.910568 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 06:38:09.911047 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4372,"mutex_wait_us":74,"thread_start_us":4177,"threads_started":1}
I20260810 06:38:09.913123 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.869054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.194:54061)
0810 06:38:09.878899 (+ 9845us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:09.878966 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:09.889227 (+ 10261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:09.889911 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:09.889915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:09.890826 (+ 911us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:09.893139 (+ 2313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:09.893159 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.894774 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.894779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:09.897355 (+ 2576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:09.897369 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.898623 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.898631 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:09.899003 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.899808 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:09.899826 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:09.902989 (+ 3163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:09.907839 (+ 4850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:09.907844 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:09.907849 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:09.908450 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:09.908903 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:09.908907 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:09.908910 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:09.909102 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:09.909851 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:09.909856 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:09.910009 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 06:38:09.912240 (+ 2231us) client_negotiation.cc:240] Negotiation successful
0810 06:38:09.912587 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9471,"mutex_wait_us":364,"spinlock_wait_cycles":39424,"thread_start_us":12923,"threads_started":1}
I20260810 06:38:09.914142 11972 heartbeater.cc:347] Connected to a master server at 127.31.59.254:36175
I20260810 06:38:09.914398 11973 heartbeater.cc:347] Connected to a master server at 127.31.59.253:37507
I20260810 06:38:09.914587 11973 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:09.914615 11972 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:09.915794 11973 heartbeater.cc:511] Master 127.31.59.253:37507 requested a full tablet report, sending...
I20260810 06:38:09.915897 11972 heartbeater.cc:511] Master 127.31.59.254:36175 requested a full tablet report, sending...
I20260810 06:38:09.917903 11122 ts_manager.cc:194] Registered new tserver with Master: 3fa769dc3836400cb8b62f521a3aad4a (127.31.59.194:36093)
I20260810 06:38:09.918001 10107 ts_manager.cc:194] Re-registered known tserver with Master: 3fa769dc3836400cb8b62f521a3aad4a (127.31.59.194:36093)
I20260810 06:38:09.919158 11978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.880609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:37537 (local address 127.31.59.194:47053)
0810 06:38:09.881596 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:09.881616 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:09.888424 (+ 6808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:09.889476 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:09.889487 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:09.890796 (+ 1309us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:09.893179 (+ 2383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:09.893189 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.894763 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.894768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:09.897355 (+ 2587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:09.897370 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.898669 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.898674 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:09.898994 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.899761 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:09.899793 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:09.902670 (+ 2877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:09.906706 (+ 4036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:09.906716 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:09.906733 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:09.907596 (+ 863us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:09.908235 (+ 639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:09.908239 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:09.908242 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:09.908472 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:09.909393 (+ 921us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:09.909401 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:09.909831 (+ 430us) client_negotiation.cc:769] Sending connection context
0810 06:38:09.918434 (+ 8603us) client_negotiation.cc:240] Negotiation successful
0810 06:38:09.918750 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"mutex_wait_us":261,"spinlock_wait_cycles":18688,"thread_start_us":554,"threads_started":1}
I20260810 06:38:09.919368 10107 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:54061
I20260810 06:38:09.920099 11970 heartbeater.cc:347] Connected to a master server at 127.31.59.252:37537
I20260810 06:38:09.920266 11970 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:09.920692 11970 heartbeater.cc:511] Master 127.31.59.252:37537 requested a full tablet report, sending...
I20260810 06:38:09.921731 11424 ts_manager.cc:194] Registered new tserver with Master: 3fa769dc3836400cb8b62f521a3aad4a (127.31.59.194:36093)
I20260810 06:38:10.004313 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 10664
I20260810 06:38:10.029508 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:44247
--local_ip_for_outbound_sockets=127.31.59.195
--tserver_master_addrs=127.31.59.254:36175
--webserver_port=34305
--webserver_interface=127.31.59.195
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.195.keytab
--principal=oryx/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:38421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.59.254:36175,127.31.59.253:37507
--tserver_master_addrs=127.31.59.254:36175,127.31.59.253:37507,127.31.59.252:37537 with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:38:10.311160 11731 heartbeater.cc:503] Master 127.31.59.254:36175 was elected leader, sending a full tablet report...
W20260810 06:38:10.354992 11985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:10.355269 11985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:10.355402 11985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:10.362406 11985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:10.362519 11985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:10.362690 11985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:38:10.372470 11985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:38421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.195.keytab
--principal=oryx/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:44247
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=34305
--tserver_master_addrs=127.31.59.254:36175,127.31.59.253:37507,127.31.59.252:37537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:10.373723 11985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:10.375020 11985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:10.387454 11995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:10.387408 11999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:10.387884 12003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:10.388329 11985 server_base.cc:1034] running on GCE node
I20260810 06:38:10.388878 11985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:10.389719 11985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:10.390908 11985 hybrid_clock.cc:648] HybridClock initialized: now 1786343890390865 us; error 30 us; skew 500 ppm
I20260810 06:38:10.391201 11985 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:10 dist-test-slave-4tdj krb5kdc[10030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343890, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:10.394040 11985 init.cc:377] Logged in from keytab as oryx/127.31.59.195@KRBTEST.COM (short username oryx)
I20260810 06:38:10.395419 11985 webserver.cc:460] Webserver started at http://127.31.59.195:34305/ using document root <none> and password file <none>
I20260810 06:38:10.396044 11985 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:10.396173 11985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:10.401263 11985 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 06:38:10.403995 12013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:10.405303 11985 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:38:10.405450 11985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "39dafaff7626463d9a8995dfdcdc2e35"
format_stamp: "Formatted at 2026-08-10 06:38:07 on dist-test-slave-4tdj"
server_key: "a53cce225b5e8287b37b472e1027a3e4"
server_key_iv: "a9abc20920a4501e1b239c20e1ffd585"
server_key_version: "encryptionkey@0"
I20260810 06:38:10.406461 11985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:10.426252 11985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:10.429797 11985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:10.430052 11985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:10.430974 11985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:10.432246 11985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:10.432340 11985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:10.432442 11985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:10.432510 11985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:10.462493 11985 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:44247
I20260810 06:38:10.462543 12192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:44247 every 8 connection(s)
I20260810 06:38:10.463855 11985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:38:10.464308 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 11985
Aug 10 06:38:10 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343890, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.252@KRBTEST.COM
Aug 10 06:38:10 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343890, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.253@KRBTEST.COM
I20260810 06:38:10.507659 12212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.472010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:55117 (local address 127.31.59.253:37507)
0810 06:38:10.480133 (+ 8123us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:10.480139 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:10.488060 (+ 7921us) server_negotiation.cc:407] Connection header received
0810 06:38:10.488234 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:10.488240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:10.488322 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:10.488433 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:10.489732 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.490928 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.494069 (+ 3141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.494688 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.494810 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.499291 (+ 4481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:10.499297 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:10.499300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:10.499360 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:10.502603 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:10.504187 (+ 1584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:10.504193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:10.504197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:10.504461 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:10.505020 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:10.505036 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:10.505040 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:10.505569 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:10.505689 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:10.506989 (+ 1300us) server_negotiation.cc:299] Negotiation successful
0810 06:38:10.507232 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7984,"thread_start_us":7872,"threads_started":1}
I20260810 06:38:10.512005 12210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.467512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:37537 (local address 127.31.59.195:39867)
0810 06:38:10.481259 (+ 13747us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:10.481325 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:10.484049 (+ 2724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:10.487695 (+ 3646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:10.487712 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:10.488565 (+ 853us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:10.489538 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:10.489558 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.491104 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.491109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:10.492377 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:10.492389 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.493541 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.493548 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:10.493839 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.494606 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:10.494637 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:10.499062 (+ 4425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:10.505818 (+ 6756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:10.505822 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:10.505825 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:10.506269 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:10.506848 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:10.506853 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:10.506856 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:10.507173 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:10.509263 (+ 2090us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:10.509267 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:10.509543 (+ 276us) client_negotiation.cc:769] Sending connection context
0810 06:38:10.509697 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 06:38:10.510341 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13415,"mutex_wait_us":49,"spinlock_wait_cycles":24320,"thread_start_us":20036,"threads_started":1}
Aug 10 06:38:10 dist-test-slave-4tdj krb5kdc[10030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343890, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM
I20260810 06:38:10.513384 12206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.468161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:39867 (local address 127.31.59.252:37537)
0810 06:38:10.487095 (+ 18934us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:10.487102 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:10.487122 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:38:10.487185 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:10.487190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:10.487275 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:10.487384 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:10.489742 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.490920 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.492520 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.493134 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.493253 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.499427 (+ 6174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:10.499434 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:10.499440 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:10.499486 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:10.505679 (+ 6193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:10.506439 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:10.506445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:10.506448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:10.506706 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:10.507311 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:10.507315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:10.507319 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:10.508821 (+ 1502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:10.508950 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:10.512557 (+ 3607us) server_negotiation.cc:299] Negotiation successful
0810 06:38:10.512788 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18804,"thread_start_us":18701,"threads_started":1}
I20260810 06:38:10.515730 12194 heartbeater.cc:347] Connected to a master server at 127.31.59.252:37537
I20260810 06:38:10.516112 12194 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:10.517068 12194 heartbeater.cc:511] Master 127.31.59.252:37537 requested a full tablet report, sending...
I20260810 06:38:10.517710 12204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.471564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:37507 (local address 127.31.59.195:55117)
0810 06:38:10.487954 (+ 16390us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:10.487971 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:10.488083 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:10.488591 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:10.488594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:10.489039 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:10.489531 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:10.489558 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.491104 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.491109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:10.493893 (+ 2784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:10.493907 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.495086 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.495093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:10.495296 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.495891 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:10.495912 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:10.499057 (+ 3145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:10.503433 (+ 4376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:10.503442 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:10.503458 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:10.504053 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:10.504619 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:10.504624 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:10.504627 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:10.504879 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:10.505881 (+ 1002us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:10.505889 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:10.506423 (+ 534us) client_negotiation.cc:769] Sending connection context
0810 06:38:10.511079 (+ 4656us) client_negotiation.cc:240] Negotiation successful
0810 06:38:10.511342 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16291,"mutex_wait_us":26,"thread_start_us":9235,"threads_started":1}
I20260810 06:38:10.518731 12198 heartbeater.cc:347] Connected to a master server at 127.31.59.253:37507
I20260810 06:38:10.518862 12198 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:10.519366 12198 heartbeater.cc:511] Master 127.31.59.253:37507 requested a full tablet report, sending...
I20260810 06:38:10.520341 11424 ts_manager.cc:194] Registered new tserver with Master: 39dafaff7626463d9a8995dfdcdc2e35 (127.31.59.195:44247)
I20260810 06:38:10.521909 11122 ts_manager.cc:194] Registered new tserver with Master: 39dafaff7626463d9a8995dfdcdc2e35 (127.31.59.195:44247)
I20260810 06:38:10.524894 12207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.468542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.195:54575)
0810 06:38:10.486128 (+ 17586us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:10.486147 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:10.486274 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:10.494755 (+ 8481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:10.494760 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:10.495464 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:10.503312 (+ 7848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:10.503326 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.504696 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.504702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:10.507543 (+ 2841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:10.507558 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.508569 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.508575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:10.508738 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.509320 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:10.509342 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:10.514328 (+ 4986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:10.519081 (+ 4753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:10.519087 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:10.519092 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:10.521279 (+ 2187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:10.522673 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:10.522679 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:10.522683 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:10.522861 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:10.523560 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:10.523565 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:10.523792 (+ 227us) client_negotiation.cc:769] Sending connection context
0810 06:38:10.523930 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 06:38:10.524254 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17465,"thread_start_us":17341,"threads_started":1}
I20260810 06:38:10.524909 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.466851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:54575 (local address 127.31.59.254:36175)
0810 06:38:10.479264 (+ 12413us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:10.479271 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:10.486247 (+ 6976us) server_negotiation.cc:407] Connection header received
0810 06:38:10.486449 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:10.486455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:10.486557 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:10.486661 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:10.503484 (+ 16823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.504579 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.507679 (+ 3100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.508396 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.508504 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.514543 (+ 6039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:10.514550 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:10.514554 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:10.514596 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:10.518569 (+ 3973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:10.521418 (+ 2849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:10.521424 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:10.521427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:10.521705 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:10.523005 (+ 1300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:10.523010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:10.523013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:10.523429 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:10.523624 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:10.524249 (+ 625us) server_negotiation.cc:299] Negotiation successful
0810 06:38:10.524501 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12149,"thread_start_us":12223,"threads_started":1}
I20260810 06:38:10.526854 12197 heartbeater.cc:347] Connected to a master server at 127.31.59.254:36175
I20260810 06:38:10.527063 12197 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:10.527583 12197 heartbeater.cc:511] Master 127.31.59.254:36175 requested a full tablet report, sending...
I20260810 06:38:10.528745 10107 ts_manager.cc:194] Re-registered known tserver with Master: 39dafaff7626463d9a8995dfdcdc2e35 (127.31.59.195:44247)
I20260810 06:38:10.529546 10107 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:54575
I20260810 06:38:10.539112 12220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.486730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47298 (local address 127.31.59.254:36175)
0810 06:38:10.511745 (+ 25015us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:10.511751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:10.515449 (+ 3698us) server_negotiation.cc:407] Connection header received
0810 06:38:10.519781 (+ 4332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:10.519788 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:10.519875 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:10.519957 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:10.524620 (+ 4663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.525731 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.527586 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.528266 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.528361 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.532363 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:10.532370 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:10.532374 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:10.532414 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:10.536330 (+ 3916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:10.536972 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:10.536978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:10.536981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:10.537113 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:10.537459 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:10.537464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:10.537468 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:10.537723 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:10.537935 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:10.538463 (+ 528us) server_negotiation.cc:299] Negotiation successful
0810 06:38:10.538688 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24854,"thread_start_us":25022,"threads_started":1}
I20260810 06:38:10.542049 12206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.497025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49238 (local address 127.31.59.252:37537)
0810 06:38:10.513667 (+ 16642us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:10.513672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:10.513689 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:38:10.513733 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:10.513738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:10.513805 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:10.513893 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:10.523662 (+ 9769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.524756 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.526693 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.527421 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.527589 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.534701 (+ 7112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:10.534708 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:10.534712 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:10.534750 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:10.538015 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:10.539280 (+ 1265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:10.539285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:10.539288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:10.539541 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:10.539956 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:10.539961 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:10.539965 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:10.540318 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:10.540653 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:10.541084 (+ 431us) server_negotiation.cc:299] Negotiation successful
0810 06:38:10.541610 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16506,"thread_start_us":17582,"threads_started":1}
I20260810 06:38:10.542474 12212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.486738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43750 (local address 127.31.59.253:37507)
0810 06:38:10.507913 (+ 21175us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:10.507919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:10.507936 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:38:10.507981 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:10.507986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:10.508068 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:10.508158 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:10.513208 (+ 5050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.514359 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.521601 (+ 7242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.522334 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.522438 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.534355 (+ 11917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:10.534363 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:10.534367 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:10.534410 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:10.537995 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:10.539154 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:10.539160 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:10.539163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:10.539354 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:10.539887 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:10.539892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:10.539895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:10.540316 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:10.540699 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:10.541795 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0810 06:38:10.541980 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21069,"thread_start_us":33982,"threads_started":1}
I20260810 06:38:10.543958 10107 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47298:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:38:10.552215 10107 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:38:10.583482 12240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.573102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:44247 (local address 127.0.0.1:58372)
0810 06:38:10.573772 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:10.573830 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:10.573945 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:10.575724 (+ 1779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:10.575728 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:10.575751 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:10.576604 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:10.576616 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.578680 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.578685 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:10.580992 (+ 2307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:10.581003 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.582822 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.582828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:10.583000 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.583034 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 06:38:10.583089 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 06:38:10.583174 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":3,"thread_start_us":424,"threads_started":1}
I20260810 06:38:10.584604 12238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.571791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:36093 (local address 127.0.0.1:35340)
0810 06:38:10.572455 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:10.572480 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:10.572607 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:10.574247 (+ 1640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:10.574253 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:10.574311 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:10.574970 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:10.574988 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.577798 (+ 2810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.577805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:10.580558 (+ 2753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:10.580573 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.583617 (+ 3044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.583626 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:10.583853 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.583885 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:38:10.583967 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 06:38:10.584099 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":33,"thread_start_us":406,"threads_started":1}
I20260810 06:38:10.584789 12241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.573399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58372 (local address 127.31.59.195:44247)
0810 06:38:10.575134 (+ 1735us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:10.575143 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:10.575184 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:38:10.575282 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:10.575290 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:10.575525 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:10.575720 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:10.576782 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.578531 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.581172 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.582600 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.583972 (+ 1372us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.584172 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:10.584276 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 06:38:10.584459 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1556,"thread_start_us":311,"threads_started":1}
I20260810 06:38:10.585522 12239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.572028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35340 (local address 127.31.59.194:36093)
0810 06:38:10.572662 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:10.572671 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:10.572703 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:38:10.572789 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:10.572795 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:10.573074 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:10.575349 (+ 2275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:10.575699 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.577594 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.580729 (+ 3135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.583290 (+ 2561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.583453 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.583749 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:10.585146 (+ 1397us) server_negotiation.cc:299] Negotiation successful
0810 06:38:10.585295 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":342,"threads_started":1}
I20260810 06:38:10.589226 12094 tablet_service.cc:1467] Processing CreateTablet for tablet e0d1a5143db9442e92f571b2a2d403c0 (DEFAULT_TABLE table=test-table [id=3b615b36c2ee4e48902e1d500c96c2df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:10.589411 11873 tablet_service.cc:1467] Processing CreateTablet for tablet e0d1a5143db9442e92f571b2a2d403c0 (DEFAULT_TABLE table=test-table [id=3b615b36c2ee4e48902e1d500c96c2df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:10.590719 12094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0d1a5143db9442e92f571b2a2d403c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:10.590894 11873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0d1a5143db9442e92f571b2a2d403c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:10.596805 12245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.573946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:33401 (local address 127.0.0.1:42924)
0810 06:38:10.575223 (+ 1277us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:10.575238 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:10.575376 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:10.588547 (+ 13171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:10.588552 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:10.588577 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:10.589487 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:10.589503 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.591825 (+ 2322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.591830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:10.593865 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:10.593881 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.596018 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.596025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:10.596207 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.596241 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 06:38:10.596335 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 06:38:10.596465 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":955,"thread_start_us":670,"threads_started":1}
I20260810 06:38:10.597007 12246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.574350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42924 (local address 127.31.59.193:33401)
0810 06:38:10.587926 (+ 13576us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:10.587936 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:10.587979 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:38:10.588096 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:10.588104 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:10.588320 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:10.588544 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:10.589705 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.591671 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.594053 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.595514 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.595675 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.595883 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:10.596413 (+ 530us) server_negotiation.cc:299] Negotiation successful
0810 06:38:10.596644 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13398,"thread_start_us":14478,"threads_started":1}
I20260810 06:38:10.600847 11610 tablet_service.cc:1467] Processing CreateTablet for tablet e0d1a5143db9442e92f571b2a2d403c0 (DEFAULT_TABLE table=test-table [id=3b615b36c2ee4e48902e1d500c96c2df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:10.604017 11610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0d1a5143db9442e92f571b2a2d403c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:10.610997 12247 tablet_bootstrap.cc:492] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35: Bootstrap starting.
I20260810 06:38:10.614045 12247 tablet_bootstrap.cc:654] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:10.615594 12247 log.cc:824] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:10.629488 12248 tablet_bootstrap.cc:492] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a: Bootstrap starting.
I20260810 06:38:10.638175 12248 tablet_bootstrap.cc:654] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:10.638444 12251 tablet_bootstrap.cc:492] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80: Bootstrap starting.
I20260810 06:38:10.640074 12248 log.cc:824] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:10.641685 12247 tablet_bootstrap.cc:492] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35: No bootstrap required, opened a new log
I20260810 06:38:10.642004 12247 ts_tablet_manager.cc:1397] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35: Time spent bootstrapping tablet: real 0.031s user 0.014s sys 0.014s
I20260810 06:38:10.643550 12251 tablet_bootstrap.cc:654] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:10.645076 12251 log.cc:824] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:10.648329 12251 tablet_bootstrap.cc:492] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80: No bootstrap required, opened a new log
I20260810 06:38:10.648644 12251 ts_tablet_manager.cc:1397] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260810 06:38:10.648777 12248 tablet_bootstrap.cc:492] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a: No bootstrap required, opened a new log
I20260810 06:38:10.649078 12248 ts_tablet_manager.cc:1397] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.006s
I20260810 06:38:10.651350 12247 raft_consensus.cc:348] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } } peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } }
I20260810 06:38:10.651753 12247 raft_consensus.cc:374] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:10.651855 12247 raft_consensus.cc:729] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39dafaff7626463d9a8995dfdcdc2e35, State: Initialized, Role: FOLLOWER
I20260810 06:38:10.652448 12247 consensus_queue.cc:260] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } } peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } }
I20260810 06:38:10.654201 12247 ts_tablet_manager.cc:1428] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35: Time spent starting tablet: real 0.012s user 0.009s sys 0.005s
I20260810 06:38:10.654524 12197 heartbeater.cc:503] Master 127.31.59.254:36175 was elected leader, sending a full tablet report...
I20260810 06:38:10.661348 12251 raft_consensus.cc:348] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } } peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } }
I20260810 06:38:10.661594 12248 raft_consensus.cc:348] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } } peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } }
I20260810 06:38:10.661880 12251 raft_consensus.cc:374] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:10.661983 12251 raft_consensus.cc:729] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7862319cb6c14028878f112d67842e80, State: Initialized, Role: FOLLOWER
I20260810 06:38:10.662051 12248 raft_consensus.cc:374] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:10.662161 12248 raft_consensus.cc:729] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fa769dc3836400cb8b62f521a3aad4a, State: Initialized, Role: FOLLOWER
I20260810 06:38:10.662693 12251 consensus_queue.cc:260] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } } peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } }
I20260810 06:38:10.662801 12248 consensus_queue.cc:260] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } } peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } }
I20260810 06:38:10.664549 12248 ts_tablet_manager.cc:1428] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a: Time spent starting tablet: real 0.015s user 0.010s sys 0.006s
I20260810 06:38:10.664673 12251 ts_tablet_manager.cc:1428] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80: Time spent starting tablet: real 0.016s user 0.009s sys 0.008s
I20260810 06:38:10.664906 11972 heartbeater.cc:503] Master 127.31.59.254:36175 was elected leader, sending a full tablet report...
W20260810 06:38:10.732936 12218 tablet.cc:2369] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:38:10.776274 11734 tablet.cc:2369] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:38:10.890489 11980 tablet.cc:2369] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:38:11.093524 12255 raft_consensus.cc:482] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:11.093788 12255 raft_consensus.cc:504] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } } peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } }
I20260810 06:38:11.095341 12255 leader_election.cc:288] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fa769dc3836400cb8b62f521a3aad4a (127.31.59.194:36093), 7862319cb6c14028878f112d67842e80 (127.31.59.193:33401)
I20260810 06:38:11.101831 12246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:11.096406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:53577 (local address 127.31.59.193:33401)
0810 06:38:11.096610 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:11.096616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:11.096947 (+ 331us) server_negotiation.cc:407] Connection header received
0810 06:38:11.097095 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:11.097100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:11.097162 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:11.097227 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:11.097648 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.098367 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.100217 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.100969 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.101134 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:11.101204 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:11.101463 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 06:38:11.101590 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:38:11.101852 12264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:11.095723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:49481 (local address 127.31.59.194:36093)
0810 06:38:11.096189 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:11.096195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:11.096217 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:38:11.096400 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:11.096406 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:11.096498 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:11.096602 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:11.097115 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.098028 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.100178 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.100963 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.101113 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:11.101186 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:11.101452 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 06:38:11.101598 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":212,"threads_started":1}
I20260810 06:38:11.101931 12260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:11.095628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:36093 (local address 127.31.59.195:49481)
0810 06:38:11.096101 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:11.096118 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:11.096256 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:11.096677 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:11.096682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:11.096727 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:11.096951 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:11.096964 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.098146 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.098152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:11.100019 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:11.100035 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.101117 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.101122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:11.101289 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:11.101330 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 06:38:11.101409 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 06:38:11.101535 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":39,"thread_start_us":194,"threads_started":1}
I20260810 06:38:11.101976 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:11.096244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:33401 (local address 127.31.59.195:53577)
0810 06:38:11.096852 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:11.096868 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:11.096967 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:11.097263 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:11.097267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:11.097292 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:11.097490 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:11.097501 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.098481 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.098494 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:11.100077 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:11.100089 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.101148 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.101153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:11.101323 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:11.101346 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 06:38:11.101421 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 06:38:11.101565 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":28,"thread_start_us":414,"threads_started":1}
I20260810 06:38:11.103056 11904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d1a5143db9442e92f571b2a2d403c0" candidate_uuid: "39dafaff7626463d9a8995dfdcdc2e35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fa769dc3836400cb8b62f521a3aad4a" is_pre_election: true
I20260810 06:38:11.103152 11647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d1a5143db9442e92f571b2a2d403c0" candidate_uuid: "39dafaff7626463d9a8995dfdcdc2e35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7862319cb6c14028878f112d67842e80" is_pre_election: true
I20260810 06:38:11.103659 11904 raft_consensus.cc:2454] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39dafaff7626463d9a8995dfdcdc2e35 in term 0.
I20260810 06:38:11.103659 11647 raft_consensus.cc:2454] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39dafaff7626463d9a8995dfdcdc2e35 in term 0.
I20260810 06:38:11.104305 12021 leader_election.cc:302] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39dafaff7626463d9a8995dfdcdc2e35, 3fa769dc3836400cb8b62f521a3aad4a; no voters:
I20260810 06:38:11.104738 12255 raft_consensus.cc:2781] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:38:11.105012 12255 raft_consensus.cc:482] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:38:11.105132 12255 raft_consensus.cc:3037] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:11.107725 12255 raft_consensus.cc:504] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } } peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } }
I20260810 06:38:11.108312 12255 leader_election.cc:288] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [CANDIDATE]: Term 1 election: Requested vote from peers 3fa769dc3836400cb8b62f521a3aad4a (127.31.59.194:36093), 7862319cb6c14028878f112d67842e80 (127.31.59.193:33401)
I20260810 06:38:11.108706 11904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d1a5143db9442e92f571b2a2d403c0" candidate_uuid: "39dafaff7626463d9a8995dfdcdc2e35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fa769dc3836400cb8b62f521a3aad4a"
I20260810 06:38:11.108804 11647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d1a5143db9442e92f571b2a2d403c0" candidate_uuid: "39dafaff7626463d9a8995dfdcdc2e35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7862319cb6c14028878f112d67842e80"
I20260810 06:38:11.108875 11904 raft_consensus.cc:3037] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:11.109001 11647 raft_consensus.cc:3037] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:11.111284 11904 raft_consensus.cc:2454] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39dafaff7626463d9a8995dfdcdc2e35 in term 1.
I20260810 06:38:11.111624 12021 leader_election.cc:302] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39dafaff7626463d9a8995dfdcdc2e35, 3fa769dc3836400cb8b62f521a3aad4a; no voters:
I20260810 06:38:11.111883 12255 raft_consensus.cc:2781] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:11.112042 11647 raft_consensus.cc:2454] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39dafaff7626463d9a8995dfdcdc2e35 in term 1.
I20260810 06:38:11.112398 12255 raft_consensus.cc:686] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [term 1 LEADER]: Becoming Leader. State: Replica: 39dafaff7626463d9a8995dfdcdc2e35, State: Running, Role: LEADER
I20260810 06:38:11.113255 12255 consensus_queue.cc:237] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } } peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } }
I20260810 06:38:11.118768 10106 catalog_manager.cc:5495] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39dafaff7626463d9a8995dfdcdc2e35 (127.31.59.195). New cstate: current_term: 1 leader_uuid: "39dafaff7626463d9a8995dfdcdc2e35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } health_report { overall_health: UNKNOWN } } }
I20260810 06:38:11.155153 12300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:11.149124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58376 (local address 127.31.59.195:44247)
0810 06:38:11.149583 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:11.149589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:11.149615 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:38:11.149781 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:11.149785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:11.149861 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:11.150001 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:11.150488 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.151186 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.152989 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.153406 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.153513 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:11.153992 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:11.154585 (+ 593us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:38:11.154718 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:11.154821 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 06:38:11.154884 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":229,"threads_started":1}
I20260810 06:38:11.163333 11904 raft_consensus.cc:1261] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a [term 1 FOLLOWER]: Refusing update from remote peer 39dafaff7626463d9a8995dfdcdc2e35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:11.163381 11647 raft_consensus.cc:1261] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80 [term 1 FOLLOWER]: Refusing update from remote peer 39dafaff7626463d9a8995dfdcdc2e35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:11.164222 12276 consensus_queue.cc:1035] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:11.164677 12255 consensus_queue.cc:1035] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:11.179550 12304 mvcc.cc:205] Tried to move back new op lower bound from 7316864578195632128 to 7316864578003578880. Current Snapshot: MvccSnapshot[applied={T|T < 7316864578195632128}]
I20260810 06:38:11.197695 10106 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:11.197986 10106 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:11.206341 10106 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f43ba0f328e4a03abc5b1f6d0226f78: Sending DeleteTablet for 3 replicas of tablet e0d1a5143db9442e92f571b2a2d403c0
I20260810 06:38:11.207588 12094 tablet_service.cc:1514] Processing DeleteTablet for tablet e0d1a5143db9442e92f571b2a2d403c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:11 UTC) from {username='oryx', principal='oryx/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:58372
I20260810 06:38:11.207610 11873 tablet_service.cc:1514] Processing DeleteTablet for tablet e0d1a5143db9442e92f571b2a2d403c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:11 UTC) from {username='oryx', principal='oryx/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:35340
I20260810 06:38:11.207752 11610 tablet_service.cc:1514] Processing DeleteTablet for tablet e0d1a5143db9442e92f571b2a2d403c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:11 UTC) from {username='oryx', principal='oryx/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:42924
I20260810 06:38:11.219571 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 10052
I20260810 06:38:11.222048 10154 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:11.241436 12321 tablet_replica.cc:330] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35: stopping tablet replica
I20260810 06:38:11.241981 12321 raft_consensus.cc:2229] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:38:11.242749 12321 raft_consensus.cc:2258] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:11.244915 12321 ts_tablet_manager.cc:1905] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:11.245934 12317 tablet_replica.cc:330] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80: stopping tablet replica
I20260810 06:38:11.246356 12318 tablet_replica.cc:330] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a: stopping tablet replica
I20260810 06:38:11.248556 12318 raft_consensus.cc:2229] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:11.250352 12317 raft_consensus.cc:2229] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:11.251993 12318 raft_consensus.cc:2258] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:11.254375 12317 raft_consensus.cc:2258] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:11.254931 12321 ts_tablet_manager.cc:1918] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:11.255146 12321 log.cc:1196] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/e0d1a5143db9442e92f571b2a2d403c0
I20260810 06:38:11.257332 12318 ts_tablet_manager.cc:1905] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:11.258836 12317 ts_tablet_manager.cc:1905] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:11.266209 12321 ts_tablet_manager.cc:1939] T e0d1a5143db9442e92f571b2a2d403c0 P 39dafaff7626463d9a8995dfdcdc2e35: Deleting consensus metadata
I20260810 06:38:11.287616 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:11.211637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43754 (local address 127.31.59.253:37507)
0810 06:38:11.238287 (+ 26650us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:11.238313 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:11.238336 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:38:11.238403 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:11.238409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:11.238490 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:11.238608 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:11.241886 (+ 3278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.243356 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.252243 (+ 8887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.253428 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.253544 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:11.273949 (+ 20405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:11.273956 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:11.273961 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:11.274003 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:11.277738 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:11.284435 (+ 6697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:11.284442 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:11.284445 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:11.284710 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:11.285341 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:11.285346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:11.285350 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:11.285805 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:11.286091 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:11.286926 (+ 835us) server_negotiation.cc:299] Negotiation successful
0810 06:38:11.287194 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26501,"thread_start_us":27123,"threads_started":1}
I20260810 06:38:11.287954 12317 ts_tablet_manager.cc:1918] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:11.288148 12317 log.cc:1196] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/e0d1a5143db9442e92f571b2a2d403c0
I20260810 06:38:11.289000 12318 ts_tablet_manager.cc:1918] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:11.289196 12318 log.cc:1196] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/e0d1a5143db9442e92f571b2a2d403c0
I20260810 06:38:11.289830 12318 ts_tablet_manager.cc:1939] T e0d1a5143db9442e92f571b2a2d403c0 P 3fa769dc3836400cb8b62f521a3aad4a: Deleting consensus metadata
I20260810 06:38:11.292253 12317 ts_tablet_manager.cc:1939] T e0d1a5143db9442e92f571b2a2d403c0 P 7862319cb6c14028878f112d67842e80: Deleting consensus metadata
I20260810 06:38:11.312466 12325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:11.218809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49246 (local address 127.31.59.252:37537)
0810 06:38:11.239339 (+ 20530us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:11.239346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:11.290502 (+ 51156us) server_negotiation.cc:407] Connection header received
0810 06:38:11.290586 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:11.290592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:11.290672 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:11.290762 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:11.294259 (+ 3497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.295805 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.297708 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.298803 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.298940 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:11.303531 (+ 4591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:11.303539 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:11.303545 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:11.303588 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:11.308268 (+ 4680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:11.309282 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:11.309288 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:11.309293 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:11.309628 (+ 335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:11.310319 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:11.310327 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:11.310330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:11.310805 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:11.311203 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:11.311862 (+ 659us) server_negotiation.cc:299] Negotiation successful
0810 06:38:11.312095 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20338,"thread_start_us":21695,"threads_started":1}
I20260810 06:38:11.415004 10080 catalog_manager.cc:4841] TS 7862319cb6c14028878f112d67842e80 (127.31.59.193:33401): tablet e0d1a5143db9442e92f571b2a2d403c0 (table test-table [id=3b615b36c2ee4e48902e1d500c96c2df]) successfully deleted
I20260810 06:38:11.415374 10080 catalog_manager.cc:4841] TS 3fa769dc3836400cb8b62f521a3aad4a (127.31.59.194:36093): tablet e0d1a5143db9442e92f571b2a2d403c0 (table test-table [id=3b615b36c2ee4e48902e1d500c96c2df]) successfully deleted
I20260810 06:38:11.415473 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 10052
I20260810 06:38:11.472198 12325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:11.450269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49248 (local address 127.31.59.252:37537)
0810 06:38:11.450488 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:11.450494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:11.457078 (+ 6584us) server_negotiation.cc:407] Connection header received
0810 06:38:11.457317 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:11.457322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:11.457388 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:11.457471 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:11.459494 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.460186 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.461769 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.462532 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.462656 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:11.465689 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:11.465696 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:11.465701 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:11.465741 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:11.468977 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:11.469741 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:11.469747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:11.469750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:11.469900 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:11.470440 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:11.470445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:11.470447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:11.470786 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:11.471008 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:11.471642 (+ 634us) server_negotiation.cc:299] Negotiation successful
0810 06:38:11.471860 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 06:38:11.475577 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:11.449518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43760 (local address 127.31.59.253:37507)
0810 06:38:11.449727 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:11.449734 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:11.451323 (+ 1589us) server_negotiation.cc:407] Connection header received
0810 06:38:11.451699 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:11.451705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:11.451805 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:11.451880 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:11.454557 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.455670 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.457691 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.458402 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.458539 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:11.469308 (+ 10769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:11.469314 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:11.469319 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:11.469361 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:11.472747 (+ 3386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:11.473503 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:11.473508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:11.473511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:11.473669 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:11.474122 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:11.474127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:11.474130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:11.474453 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:11.474729 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:11.475016 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 06:38:11.475229 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 06:38:12.186934 12340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.186215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.193:50903)
0810 06:38:12.186662 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.186770 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":211,"threads_started":1}
W20260810 06:38:12.187407 11731 heartbeater.cc:644] Failed to heartbeat to 127.31.59.254:36175 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
I20260810 06:38:12.193373 12344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.192462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.194:39633)
0810 06:38:12.193040 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.193185 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":361,"threads_started":1}
I20260810 06:38:12.193986 12341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.192128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.195:57851)
0810 06:38:12.193686 (+ 1558us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.193817 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1454,"thread_start_us":1385,"threads_started":1}
W20260810 06:38:12.194141 11972 heartbeater.cc:644] Failed to heartbeat to 127.31.59.254:36175 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
W20260810 06:38:12.194391 12197 heartbeater.cc:644] Failed to heartbeat to 127.31.59.254:36175 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
I20260810 06:38:12.705554 12347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f43ba0f328e4a03abc5b1f6d0226f78)
I20260810 06:38:12.705564 12346 raft_consensus.cc:482] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f43ba0f328e4a03abc5b1f6d0226f78)
I20260810 06:38:12.705822 12347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false } }
I20260810 06:38:12.705843 12346 raft_consensus.cc:504] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false } }
I20260810 06:38:12.707233 12347 leader_election.cc:288] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f43ba0f328e4a03abc5b1f6d0226f78 (127.31.59.254:36175), a63e2108523a4d96ac09190af21127a0 (127.31.59.253:37507)
I20260810 06:38:12.718221 12346 leader_election.cc:288] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f43ba0f328e4a03abc5b1f6d0226f78 (127.31.59.254:36175), 5b6624b357734638a9272097b85844c9 (127.31.59.252:37537)
I20260810 06:38:12.720391 12354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.707343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.0.0.1:47736)
0810 06:38:12.719904 (+ 12561us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.720112 (+ 208us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12425,"threads_started":1}
W20260810 06:38:12.721680 11401 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
I20260810 06:38:12.722332 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.707355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.0.0.1:47734)
0810 06:38:12.721799 (+ 14444us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.722025 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14340,"thread_start_us":14203,"threads_started":1}
W20260810 06:38:12.724151 11102 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
I20260810 06:38:12.724985 12352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.724336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.0.0.1:47748)
0810 06:38:12.724520 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.724729 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 06:38:12.725324 11401 leader_election.cc:334] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4f43ba0f328e4a03abc5b1f6d0226f78 (127.31.59.254:36175): Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
I20260810 06:38:12.727308 12353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.726665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.0.0.1:47750)
0810 06:38:12.726856 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.727049 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 06:38:12.727502 11102 leader_election.cc:334] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4f43ba0f328e4a03abc5b1f6d0226f78 (127.31.59.254:36175): Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
I20260810 06:38:12.729442 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.707385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:37537 (local address 127.0.0.1:49262)
0810 06:38:12.722609 (+ 15224us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.722633 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:12.722833 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:12.723447 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:12.723452 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:12.723480 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:12.723762 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.723774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.725713 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.725720 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.727629 (+ 1909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.727643 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.728793 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.728799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.728929 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.728958 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 06:38:12.729045 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 06:38:12.729217 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15112,"mutex_wait_us":59,"thread_start_us":15897,"threads_started":1}
I20260810 06:38:12.729544 12358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.707833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49262 (local address 127.31.59.252:37537)
0810 06:38:12.717107 (+ 9274us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:12.717114 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:12.722717 (+ 5603us) server_negotiation.cc:407] Connection header received
0810 06:38:12.722980 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:12.722987 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:12.723059 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:12.723128 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:12.723982 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.725487 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.727797 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.728641 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.728738 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.728828 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:12.729089 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 06:38:12.729219 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":9143,"thread_start_us":8962,"threads_started":1}
I20260810 06:38:12.730147 11441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a63e2108523a4d96ac09190af21127a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5b6624b357734638a9272097b85844c9" is_pre_election: true
I20260810 06:38:12.730564 11441 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a63e2108523a4d96ac09190af21127a0 in term 1.
I20260810 06:38:12.731022 12354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.707752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:37507 (local address 127.0.0.1:43772)
0810 06:38:12.720864 (+ 13112us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.720890 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:12.721033 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:12.724857 (+ 3824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:12.724862 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:12.724915 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:12.725275 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.725291 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.727165 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.727170 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.728697 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.728712 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.730235 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.730242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.730435 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.730466 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:38:12.730583 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 06:38:12.730713 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13019,"mutex_wait_us":1,"thread_start_us":12810,"threads_started":1}
I20260810 06:38:12.731062 12359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.721799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43772 (local address 127.31.59.253:37507)
0810 06:38:12.724537 (+ 2738us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:12.724543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:12.724560 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:38:12.724619 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:12.724623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:12.724691 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:12.725432 (+ 741us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:12.725747 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.727024 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.728868 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.730057 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.730148 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.730217 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:12.730604 (+ 387us) server_negotiation.cc:299] Negotiation successful
0810 06:38:12.730714 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":2274,"thread_start_us":2091,"threads_started":1}
I20260810 06:38:12.731112 11096 leader_election.cc:302] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b6624b357734638a9272097b85844c9, a63e2108523a4d96ac09190af21127a0; no voters: 4f43ba0f328e4a03abc5b1f6d0226f78
I20260810 06:38:12.731528 12346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 06:38:12.731673 12346 raft_consensus.cc:482] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f43ba0f328e4a03abc5b1f6d0226f78)
I20260810 06:38:12.731624 11133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5b6624b357734638a9272097b85844c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a63e2108523a4d96ac09190af21127a0" is_pre_election: true
I20260810 06:38:12.731784 12346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:38:12.733848 12346 raft_consensus.cc:504] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false } }
I20260810 06:38:12.734261 11133 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b6624b357734638a9272097b85844c9 in current term 2: Already voted for candidate a63e2108523a4d96ac09190af21127a0 in this term.
I20260810 06:38:12.734573 12346 leader_election.cc:288] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [CANDIDATE]: Term 2 election: Requested vote from peers 4f43ba0f328e4a03abc5b1f6d0226f78 (127.31.59.254:36175), 5b6624b357734638a9272097b85844c9 (127.31.59.252:37537)
I20260810 06:38:12.734956 11441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a63e2108523a4d96ac09190af21127a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5b6624b357734638a9272097b85844c9"
I20260810 06:38:12.735227 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.734859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.0.0.1:47762)
0810 06:38:12.735001 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.735086 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 06:38:12.735244 11441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:38:12.736037 11400 leader_election.cc:302] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b6624b357734638a9272097b85844c9; no voters: 4f43ba0f328e4a03abc5b1f6d0226f78, a63e2108523a4d96ac09190af21127a0
I20260810 06:38:12.736989 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.736327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.0.0.1:47770)
0810 06:38:12.736776 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.736857 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295}
W20260810 06:38:12.737166 11102 leader_election.cc:334] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4f43ba0f328e4a03abc5b1f6d0226f78 (127.31.59.254:36175): Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
I20260810 06:38:12.740232 11441 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a63e2108523a4d96ac09190af21127a0 in term 2.
I20260810 06:38:12.740427 12347 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:38:12.740604 11096 leader_election.cc:302] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b6624b357734638a9272097b85844c9, a63e2108523a4d96ac09190af21127a0; no voters: 4f43ba0f328e4a03abc5b1f6d0226f78
I20260810 06:38:12.741158 12346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:38:12.742748 12346 raft_consensus.cc:686] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [term 2 LEADER]: Becoming Leader. State: Replica: a63e2108523a4d96ac09190af21127a0, State: Running, Role: LEADER
I20260810 06:38:12.743185 12346 consensus_queue.cc:237] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false } }
I20260810 06:38:12.748345 12363 sys_catalog.cc:455] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a63e2108523a4d96ac09190af21127a0. Latest consensus state: current_term: 2 leader_uuid: "a63e2108523a4d96ac09190af21127a0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false } } }
I20260810 06:38:12.748715 12363 sys_catalog.cc:458] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:12.749430 12365 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:38:12.753413 12365 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:38:12.754186 12365 catalog_manager.cc:1259] Loaded cluster ID: 1a7d2e9180204d73b144bbdde26e2e50
I20260810 06:38:12.754264 12365 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:38:12.755369 12365 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:38:12.756145 12365 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: Loaded TSK: 0
I20260810 06:38:12.757239 12365 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:38:12.806700 11122 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:38:12.816843 11122 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:38:12.821007 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.820590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.0.0.1:47800)
0810 06:38:12.820793 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.820871 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 06:38:12.821039 11441 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [term 2 FOLLOWER]: Refusing update from remote peer a63e2108523a4d96ac09190af21127a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260810 06:38:12.821826 12363 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260810 06:38:12.822117 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.821719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.0.0.1:47814)
0810 06:38:12.821871 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.821977 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 06:38:12.822314 11102 consensus_peers.cc:487] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 -> Peer 4f43ba0f328e4a03abc5b1f6d0226f78 (127.31.59.254:36175): Couldn't send request to peer 4f43ba0f328e4a03abc5b1f6d0226f78. Status: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 06:38:12.824451 12347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a63e2108523a4d96ac09190af21127a0. Latest consensus state: current_term: 2 leader_uuid: "a63e2108523a4d96ac09190af21127a0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false } } }
I20260810 06:38:12.824792 12347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:38:12.826112 12363 sys_catalog.cc:455] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a63e2108523a4d96ac09190af21127a0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false } } }
I20260810 06:38:12.826419 12363 sys_catalog.cc:458] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:12.826695 12367 mvcc.cc:205] Tried to move back new op lower bound from 7316864584988393472 to 7316864584680308736. Current Snapshot: MvccSnapshot[applied={T|T < 7316864584988393472}]
I20260810 06:38:12.834811 12347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a63e2108523a4d96ac09190af21127a0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4f43ba0f328e4a03abc5b1f6d0226f78" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36175 } } peers { permanent_uuid: "a63e2108523a4d96ac09190af21127a0" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 37507 } attrs { promote: false } } peers { permanent_uuid: "5b6624b357734638a9272097b85844c9" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 37537 } attrs { promote: false } } }
I20260810 06:38:12.835074 12347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b6624b357734638a9272097b85844c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:38:12.846755 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.838897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:33401 (local address 127.0.0.1:42928)
0810 06:38:12.839166 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.839185 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:12.839305 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:12.840195 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:12.840201 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:12.840230 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:12.840472 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.840483 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.842217 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.842223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.844113 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.844129 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.845878 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.845887 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.846116 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.846155 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 06:38:12.846247 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 06:38:12.846424 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":1}
I20260810 06:38:12.847652 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.839078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42928 (local address 127.31.59.193:33401)
0810 06:38:12.839871 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:12.839877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:12.839895 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:38:12.839963 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:12.839968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:12.840038 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:12.840662 (+ 624us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:12.840836 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.842079 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.844317 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.845539 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.845662 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.845774 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:12.847234 (+ 1460us) server_negotiation.cc:299] Negotiation successful
0810 06:38:12.847376 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":552,"threads_started":1}
I20260810 06:38:12.848304 11610 tablet_service.cc:1467] Processing CreateTablet for tablet d693829fe897484b98e35a6bf08f22f6 (DEFAULT_TABLE table=test-table [id=a4be35364cb343649ba395c87e14a5f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:12.848753 11610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d693829fe897484b98e35a6bf08f22f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:12.849280 12353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.839325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:36093 (local address 127.0.0.1:35356)
0810 06:38:12.839641 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.839664 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:12.839770 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:12.843978 (+ 4208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:12.843982 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:12.844018 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:12.844412 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.844424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.845618 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.845623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.847210 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.847223 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.848566 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.848571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.848733 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.848764 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:38:12.848852 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 06:38:12.848980 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212}
I20260810 06:38:12.849983 12379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.839992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35356 (local address 127.31.59.194:36093)
0810 06:38:12.843681 (+ 3689us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:12.843686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:12.843704 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:38:12.843783 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:12.843787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:12.843856 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:12.843933 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:12.844584 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.845489 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.847361 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.848336 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.848471 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.848560 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:12.849571 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0810 06:38:12.849708 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3582,"thread_start_us":3469,"threads_started":1}
I20260810 06:38:12.850407 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.839703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:44247 (local address 127.0.0.1:58390)
0810 06:38:12.843142 (+ 3439us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.843161 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:12.843296 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:12.843832 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:12.843837 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:12.843860 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:12.844310 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.844321 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.845699 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.845704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.847799 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.847814 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.849607 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.849613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.849782 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.849814 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:38:12.849900 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 06:38:12.850016 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3318,"mutex_wait_us":20,"thread_start_us":3151,"threads_started":1}
I20260810 06:38:12.850821 12380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.840068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58390 (local address 127.31.59.195:44247)
0810 06:38:12.842621 (+ 2553us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:12.842628 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:12.843256 (+ 628us) server_negotiation.cc:407] Connection header received
0810 06:38:12.843415 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:12.843420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:12.843492 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:12.843594 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:12.844496 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.845530 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.847954 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.849257 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.849374 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.849467 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:12.850394 (+ 927us) server_negotiation.cc:299] Negotiation successful
0810 06:38:12.850542 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2399,"thread_start_us":2284,"threads_started":1}
I20260810 06:38:12.850948 11873 tablet_service.cc:1467] Processing CreateTablet for tablet d693829fe897484b98e35a6bf08f22f6 (DEFAULT_TABLE table=test-table [id=a4be35364cb343649ba395c87e14a5f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:12.851341 11873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d693829fe897484b98e35a6bf08f22f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:12.854331 12382 tablet_bootstrap.cc:492] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80: Bootstrap starting.
I20260810 06:38:12.854468 12094 tablet_service.cc:1467] Processing CreateTablet for tablet d693829fe897484b98e35a6bf08f22f6 (DEFAULT_TABLE table=test-table [id=a4be35364cb343649ba395c87e14a5f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:12.854878 12094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d693829fe897484b98e35a6bf08f22f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:12.857331 12382 tablet_bootstrap.cc:654] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:12.859699 12382 tablet_bootstrap.cc:492] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80: No bootstrap required, opened a new log
I20260810 06:38:12.859853 12382 ts_tablet_manager.cc:1397] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260810 06:38:12.860366 12386 tablet_bootstrap.cc:492] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35: Bootstrap starting.
I20260810 06:38:12.860679 12382 raft_consensus.cc:348] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } } peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } }
I20260810 06:38:12.860941 12382 raft_consensus.cc:374] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:12.861042 12382 raft_consensus.cc:729] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7862319cb6c14028878f112d67842e80, State: Initialized, Role: FOLLOWER
I20260810 06:38:12.861296 12382 consensus_queue.cc:260] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } } peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } }
I20260810 06:38:12.861804 12386 tablet_bootstrap.cc:654] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:12.863605 12383 tablet_bootstrap.cc:492] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a: Bootstrap starting.
I20260810 06:38:12.866444 12383 tablet_bootstrap.cc:654] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:12.867228 12382 ts_tablet_manager.cc:1428] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80: Time spent starting tablet: real 0.007s user 0.000s sys 0.003s
I20260810 06:38:12.869052 11733 heartbeater.cc:503] Master 127.31.59.253:37507 was elected leader, sending a full tablet report...
I20260810 06:38:12.872263 12388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.862425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.193:40439)
0810 06:38:12.871806 (+ 9381us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.871975 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9235,"thread_start_us":9099,"threads_started":1}
I20260810 06:38:12.873214 12388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.872836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.193:41541)
0810 06:38:12.872985 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.873083 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 06:38:12.874537 12386 tablet_bootstrap.cc:492] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35: No bootstrap required, opened a new log
I20260810 06:38:12.874691 12386 ts_tablet_manager.cc:1397] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260810 06:38:12.874753 12383 tablet_bootstrap.cc:492] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a: No bootstrap required, opened a new log
I20260810 06:38:12.874893 12383 ts_tablet_manager.cc:1397] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260810 06:38:12.875481 12386 raft_consensus.cc:348] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } } peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } }
I20260810 06:38:12.875674 12386 raft_consensus.cc:374] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:12.875770 12386 raft_consensus.cc:729] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39dafaff7626463d9a8995dfdcdc2e35, State: Initialized, Role: FOLLOWER
I20260810 06:38:12.875718 12383 raft_consensus.cc:348] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } } peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } }
I20260810 06:38:12.875955 12383 raft_consensus.cc:374] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:12.876040 12383 raft_consensus.cc:729] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fa769dc3836400cb8b62f521a3aad4a, State: Initialized, Role: FOLLOWER
I20260810 06:38:12.876012 12386 consensus_queue.cc:260] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } } peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } }
I20260810 06:38:12.876255 12383 consensus_queue.cc:260] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } } peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } }
I20260810 06:38:12.876858 12386 ts_tablet_manager.cc:1428] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:38:12.878348 12394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.877497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.195:43397)
0810 06:38:12.877959 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.878103 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"threads_started":1}
I20260810 06:38:12.880029 12394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.879664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.195:53763)
0810 06:38:12.879786 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.879879 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260810 06:38:12.880579 12198 heartbeater.cc:503] Master 127.31.59.253:37507 was elected leader, sending a full tablet report...
I20260810 06:38:12.881429 12383 ts_tablet_manager.cc:1428] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260810 06:38:12.886471 11973 heartbeater.cc:503] Master 127.31.59.253:37507 was elected leader, sending a full tablet report...
I20260810 06:38:12.887180 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.881615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.194:49213)
0810 06:38:12.886812 (+ 5197us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.886986 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5058,"thread_start_us":4980,"threads_started":1}
I20260810 06:38:12.888190 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.887822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.194:51925)
0810 06:38:12.887953 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.888035 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 06:38:12.909547 12387 raft_consensus.cc:482] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:12.909792 12387 raft_consensus.cc:504] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } } peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } }
I20260810 06:38:12.911369 12387 leader_election.cc:288] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fa769dc3836400cb8b62f521a3aad4a (127.31.59.194:36093), 39dafaff7626463d9a8995dfdcdc2e35 (127.31.59.195:44247)
I20260810 06:38:12.917335 12379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.911696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:54403 (local address 127.31.59.194:36093)
0810 06:38:12.911887 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:12.911893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:12.911912 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:38:12.912104 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:12.912109 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:12.912186 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:12.912269 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:12.912828 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.913891 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.915705 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.916464 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.916620 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.916701 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:12.916972 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 06:38:12.917099 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:38:12.917361 12388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.911570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:36093 (local address 127.31.59.193:54403)
0810 06:38:12.911761 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.911794 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:12.911956 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:12.912326 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:12.912335 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:12.912371 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:12.912671 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.912684 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.914023 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.914028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.915554 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.915566 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.916676 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.916682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.916831 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.916855 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 06:38:12.916935 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:38:12.917090 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 06:38:12.917637 12380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.912018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:54995 (local address 127.31.59.195:44247)
0810 06:38:12.912222 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:12.912228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:12.912597 (+ 369us) server_negotiation.cc:407] Connection header received
0810 06:38:12.912708 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:12.912713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:12.912786 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:12.912862 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:12.913497 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.914414 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.916053 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.916785 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.916924 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.916992 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:12.917294 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 06:38:12.917412 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:38:12.917703 12397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.911947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:44247 (local address 127.31.59.193:54995)
0810 06:38:12.912483 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.912503 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:12.912604 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:12.912899 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:12.912903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:12.912929 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:12.913270 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.913283 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.914552 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.914557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.915909 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.915920 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.917015 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.917022 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.917154 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.917177 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 06:38:12.917252 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 06:38:12.917376 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":274,"threads_started":1}
I20260810 06:38:12.918146 11904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d693829fe897484b98e35a6bf08f22f6" candidate_uuid: "7862319cb6c14028878f112d67842e80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fa769dc3836400cb8b62f521a3aad4a" is_pre_election: true
I20260810 06:38:12.918428 11904 raft_consensus.cc:2454] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7862319cb6c14028878f112d67842e80 in term 0.
I20260810 06:38:12.918678 12129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d693829fe897484b98e35a6bf08f22f6" candidate_uuid: "7862319cb6c14028878f112d67842e80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39dafaff7626463d9a8995dfdcdc2e35" is_pre_election: true
I20260810 06:38:12.918892 11539 leader_election.cc:302] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fa769dc3836400cb8b62f521a3aad4a, 7862319cb6c14028878f112d67842e80; no voters:
I20260810 06:38:12.919073 12129 raft_consensus.cc:2454] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7862319cb6c14028878f112d67842e80 in term 0.
I20260810 06:38:12.919239 12387 raft_consensus.cc:2781] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:38:12.919348 12387 raft_consensus.cc:482] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:38:12.919461 12387 raft_consensus.cc:3037] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:12.921554 12387 raft_consensus.cc:504] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } } peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } }
I20260810 06:38:12.922065 12387 leader_election.cc:288] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [CANDIDATE]: Term 1 election: Requested vote from peers 3fa769dc3836400cb8b62f521a3aad4a (127.31.59.194:36093), 39dafaff7626463d9a8995dfdcdc2e35 (127.31.59.195:44247)
I20260810 06:38:12.922500 11904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d693829fe897484b98e35a6bf08f22f6" candidate_uuid: "7862319cb6c14028878f112d67842e80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fa769dc3836400cb8b62f521a3aad4a"
I20260810 06:38:12.922596 12129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d693829fe897484b98e35a6bf08f22f6" candidate_uuid: "7862319cb6c14028878f112d67842e80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39dafaff7626463d9a8995dfdcdc2e35"
I20260810 06:38:12.922676 11904 raft_consensus.cc:3037] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:12.922770 12129 raft_consensus.cc:3037] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:12.924573 11904 raft_consensus.cc:2454] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7862319cb6c14028878f112d67842e80 in term 1.
I20260810 06:38:12.924769 12129 raft_consensus.cc:2454] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7862319cb6c14028878f112d67842e80 in term 1.
I20260810 06:38:12.924898 11539 leader_election.cc:302] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fa769dc3836400cb8b62f521a3aad4a, 7862319cb6c14028878f112d67842e80; no voters:
I20260810 06:38:12.925184 12387 raft_consensus.cc:2781] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:12.925601 12387 raft_consensus.cc:686] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [term 1 LEADER]: Becoming Leader. State: Replica: 7862319cb6c14028878f112d67842e80, State: Running, Role: LEADER
I20260810 06:38:12.925997 12387 consensus_queue.cc:237] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } } peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } }
I20260810 06:38:12.926471 12388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.926069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.193:38029)
0810 06:38:12.926217 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.926328 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 06:38:12.929700 11121 catalog_manager.cc:5495] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7862319cb6c14028878f112d67842e80 (127.31.59.193). New cstate: current_term: 1 leader_uuid: "7862319cb6c14028878f112d67842e80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7862319cb6c14028878f112d67842e80" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 } health_report { overall_health: UNKNOWN } } }
I20260810 06:38:12.968768 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.964025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42936 (local address 127.31.59.193:33401)
0810 06:38:12.964241 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:12.964245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:12.964263 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:38:12.964487 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:12.964492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:12.964593 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:12.964717 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:12.965040 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.965746 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.966996 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.967406 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.967496 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.967841 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:12.968345 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:38:12.968454 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:12.968490 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 06:38:12.968550 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:38:12.972790 11904 raft_consensus.cc:1261] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a [term 1 FOLLOWER]: Refusing update from remote peer 7862319cb6c14028878f112d67842e80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:12.973026 12129 raft_consensus.cc:1261] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35 [term 1 FOLLOWER]: Refusing update from remote peer 7862319cb6c14028878f112d67842e80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:12.973467 12398 consensus_queue.cc:1035] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fa769dc3836400cb8b62f521a3aad4a" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:12.973876 12387 consensus_queue.cc:1035] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39dafaff7626463d9a8995dfdcdc2e35" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 44247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:12.976472 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.976031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.194:46831)
0810 06:38:12.976186 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.976295 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 06:38:12.976822 12394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.975935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.195:42303)
0810 06:38:12.976485 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.976600 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414}
I20260810 06:38:12.984582 12394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.981981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.195:41689)
0810 06:38:12.984351 (+ 2370us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.984438 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2262}
I20260810 06:38:12.984627 12408 mvcc.cc:205] Tried to move back new op lower bound from 7316864585609453568 to 7316864585428865024. Current Snapshot: MvccSnapshot[applied={T|T < 7316864585609453568}]
I20260810 06:38:12.986202 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.985704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.194:46197)
0810 06:38:12.985932 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.986027 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260810 06:38:12.987212 12388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.986767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.31.59.193:43573)
0810 06:38:12.986944 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.987028 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 06:38:12.994897 11122 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:12.995179 11122 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:13.001572 11122 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a63e2108523a4d96ac09190af21127a0: Sending DeleteTablet for 3 replicas of tablet d693829fe897484b98e35a6bf08f22f6
I20260810 06:38:13.002566 11610 tablet_service.cc:1514] Processing DeleteTablet for tablet d693829fe897484b98e35a6bf08f22f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:12 UTC) from {username='oryx', principal='oryx/127.31.59.253@KRBTEST.COM'} at 127.0.0.1:42928
I20260810 06:38:13.002658 12094 tablet_service.cc:1514] Processing DeleteTablet for tablet d693829fe897484b98e35a6bf08f22f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:12 UTC) from {username='oryx', principal='oryx/127.31.59.253@KRBTEST.COM'} at 127.0.0.1:58390
I20260810 06:38:13.002849 11873 tablet_service.cc:1514] Processing DeleteTablet for tablet d693829fe897484b98e35a6bf08f22f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:12 UTC) from {username='oryx', principal='oryx/127.31.59.253@KRBTEST.COM'} at 127.0.0.1:35356
I20260810 06:38:13.004263 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 11397
I20260810 06:38:13.013470 12423 tablet_replica.cc:330] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a: stopping tablet replica
I20260810 06:38:13.013689 12423 raft_consensus.cc:2229] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:13.014029 12423 raft_consensus.cc:2258] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:13.015379 12418 tablet_replica.cc:330] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80: stopping tablet replica
I20260810 06:38:13.015678 12418 raft_consensus.cc:2229] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:38:13.016417 12418 raft_consensus.cc:2258] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:13.016495 12423 ts_tablet_manager.cc:1905] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:13.017625 12418 ts_tablet_manager.cc:1905] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:13.028119 12422 tablet_replica.cc:330] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35: stopping tablet replica
I20260810 06:38:13.028339 12422 raft_consensus.cc:2229] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:13.028692 12422 raft_consensus.cc:2258] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:13.030586 12418 ts_tablet_manager.cc:1918] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:13.031203 12418 log.cc:1196] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/d693829fe897484b98e35a6bf08f22f6
I20260810 06:38:13.032253 12423 ts_tablet_manager.cc:1918] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:13.032397 12423 log.cc:1196] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/d693829fe897484b98e35a6bf08f22f6
I20260810 06:38:13.032847 12423 ts_tablet_manager.cc:1939] T d693829fe897484b98e35a6bf08f22f6 P 3fa769dc3836400cb8b62f521a3aad4a: Deleting consensus metadata
I20260810 06:38:13.033239 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:13.005642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42942 (local address 127.31.59.193:33401)
0810 06:38:13.005811 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:13.005817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:13.011858 (+ 6041us) server_negotiation.cc:407] Connection header received
0810 06:38:13.012032 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:13.012038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:13.012147 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:13.012246 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:13.014885 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:13.016189 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:13.018116 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:13.018801 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:13.018930 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:13.022244 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:13.022265 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:13.022286 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:13.022364 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:13.026664 (+ 4300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:13.030848 (+ 4184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:13.030859 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:13.030863 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:13.031052 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:13.031466 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:13.031483 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:13.031486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:13.031957 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:13.032278 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:13.032613 (+ 335us) server_negotiation.cc:299] Negotiation successful
0810 06:38:13.032836 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:38:13.034109 12418 ts_tablet_manager.cc:1939] T d693829fe897484b98e35a6bf08f22f6 P 7862319cb6c14028878f112d67842e80: Deleting consensus metadata
I20260810 06:38:13.034555 11102 catalog_manager.cc:4841] TS 3fa769dc3836400cb8b62f521a3aad4a (127.31.59.194:36093): tablet d693829fe897484b98e35a6bf08f22f6 (table test-table [id=a4be35364cb343649ba395c87e14a5f3]) successfully deleted
I20260810 06:38:13.034142 11723 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:13.035120 12422 ts_tablet_manager.cc:1905] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:13.035533 11102 catalog_manager.cc:4841] TS 7862319cb6c14028878f112d67842e80 (127.31.59.193:33401): tablet d693829fe897484b98e35a6bf08f22f6 (table test-table [id=a4be35364cb343649ba395c87e14a5f3]) successfully deleted
I20260810 06:38:13.042551 12422 ts_tablet_manager.cc:1918] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:13.042721 12422 log.cc:1196] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/d693829fe897484b98e35a6bf08f22f6
I20260810 06:38:13.043337 12422 ts_tablet_manager.cc:1939] T d693829fe897484b98e35a6bf08f22f6 P 39dafaff7626463d9a8995dfdcdc2e35: Deleting consensus metadata
I20260810 06:38:13.044700 11102 catalog_manager.cc:4841] TS 39dafaff7626463d9a8995dfdcdc2e35 (127.31.59.195:44247): tablet d693829fe897484b98e35a6bf08f22f6 (table test-table [id=a4be35364cb343649ba395c87e14a5f3]) successfully deleted
I20260810 06:38:13.147856 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 11397
I20260810 06:38:13.172876 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 11748
I20260810 06:38:13.186239 12379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:13.173618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35372 (local address 127.31.59.194:36093)
0810 06:38:13.173833 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:13.173837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:13.173856 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:38:13.174022 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:13.174026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:13.174106 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:13.174188 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:13.175521 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:13.176324 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:13.177580 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:13.178017 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:13.178126 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:13.180747 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:13.180765 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:13.180783 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:13.180818 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:13.183632 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:13.184290 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:13.184298 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:13.184301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:13.184444 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:13.184794 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:13.184798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:13.184800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:13.185194 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:13.185501 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:13.185683 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 06:38:13.185917 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:38:13.187197 11963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:13.276376 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:13.275860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.0.0.1:47820)
0810 06:38:13.276057 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:13.276170 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 06:38:13.277474 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:13.277057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36175 (local address 127.0.0.1:47832)
0810 06:38:13.277227 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:13.277328 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:36175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 06:38:13.278466 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 11748
I20260810 06:38:13.305954 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 11985
I20260810 06:38:13.318982 12380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:13.306687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58396 (local address 127.31.59.195:44247)
0810 06:38:13.306849 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:13.306853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:13.306914 (+ 61us) server_negotiation.cc:407] Connection header received
0810 06:38:13.307086 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:13.307089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:13.307164 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:13.307239 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:13.308573 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:13.309294 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:13.310547 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:13.311004 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:13.311098 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:13.313420 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:13.313437 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:13.313455 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:13.313521 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:13.316545 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:13.317209 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:13.317217 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:13.317220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:13.317348 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:13.317654 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:13.317657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:13.317658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:13.318024 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:13.318267 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:13.318569 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 06:38:13.318736 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:38:13.319634 12188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:13.436228 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 11985
I20260810 06:38:13.464277 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 10902
I20260810 06:38:13.465452 11168 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:13.589265 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 10902
W20260810 06:38:13.615046 11399 connection.cc:535] server connection from 127.0.0.1:49262 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 06:38:13.615381 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 11060
I20260810 06:38:13.616420 11474 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:13.731910 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 11060
2026-08-10T06:38:13Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9971 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:38:13 dist-test-slave-4tdj krb5kdc[12435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:38:13 dist-test-slave-4tdj krb5kdc[12435](info): set up 2 sockets
Aug 10 06:38:13 dist-test-slave-4tdj krb5kdc[12435](info): commencing operation
krb5kdc: starting...
W20260810 06:38:14.951937 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.006s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:38:15 dist-test-slave-4tdj krb5kdc[12435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:38:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:38:15Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.254.keytab.
Entry for principal oryx/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.254.keytab.
I20260810 06:38:15.164819 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40097
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.254.keytab
--principal=oryx/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:40097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:15.482749 12453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:15.483018 12453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:15.483142 12453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:15.490190 12453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:15.490305 12453 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 06:38:15.490376 12453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:15.490461 12453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:15.490525 12453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:15.490607 12453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:15.500231 12453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:40097
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.254.keytab
--principal=oryx/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:15.501472 12453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:15.502859 12453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:15.514397 12463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:15.514370 12464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:15.514436 12467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:15.514946 12453 server_base.cc:1034] running on GCE node
I20260810 06:38:15.515738 12453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:15.517082 12453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:15.518285 12453 hybrid_clock.cc:648] HybridClock initialized: now 1786343895518218 us; error 50 us; skew 500 ppm
I20260810 06:38:15.518622 12453 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:15 dist-test-slave-4tdj krb5kdc[12435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343895, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:15.521545 12453 init.cc:377] Logged in from keytab as oryx/127.31.59.254@KRBTEST.COM (short username oryx)
I20260810 06:38:15.523025 12453 webserver.cc:460] Webserver started at http://127.31.59.254:42613/ using document root <none> and password file <none>
I20260810 06:38:15.523675 12453 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:15.523774 12453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:15.524073 12453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:15.526952 12453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "45a743dbe05b4058a24dfe33b83883cc"
format_stamp: "Formatted at 2026-08-10 06:38:15 on dist-test-slave-4tdj"
server_key: "c21fe43ae233514aa40c74d57fe8d48c"
server_key_iv: "03abbb6d170f8a0e62625b8a413abcf6"
server_key_version: "encryptionkey@0"
I20260810 06:38:15.527699 12453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "45a743dbe05b4058a24dfe33b83883cc"
format_stamp: "Formatted at 2026-08-10 06:38:15 on dist-test-slave-4tdj"
server_key: "c21fe43ae233514aa40c74d57fe8d48c"
server_key_iv: "03abbb6d170f8a0e62625b8a413abcf6"
server_key_version: "encryptionkey@0"
I20260810 06:38:15.532541 12453 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:38:15.535775 12479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:15.536978 12453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:38:15.537120 12453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "45a743dbe05b4058a24dfe33b83883cc"
format_stamp: "Formatted at 2026-08-10 06:38:15 on dist-test-slave-4tdj"
server_key: "c21fe43ae233514aa40c74d57fe8d48c"
server_key_iv: "03abbb6d170f8a0e62625b8a413abcf6"
server_key_version: "encryptionkey@0"
I20260810 06:38:15.537312 12453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:15.573556 12453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:15.577643 12453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:15.577893 12453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:15.597500 12453 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:40097
I20260810 06:38:15.597524 12577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:40097 every 8 connection(s)
I20260810 06:38:15.598835 12453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:38:15.602686 12580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:15.603830 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 12453
I20260810 06:38:15.604072 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:38:15.604894 31983 external_mini_cluster.cc:1428] Setting key e835ce10c8197b608e265eff55c2fea6
I20260810 06:38:15.615334 12580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc: Bootstrap starting.
Aug 10 06:38:15 dist-test-slave-4tdj krb5kdc[12435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM
I20260810 06:38:15.618475 12580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:15.619553 12580 log.cc:824] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:15.622442 12580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc: No bootstrap required, opened a new log
I20260810 06:38:15.625908 12586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.607032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56650 (local address 127.31.59.254:40097)
0810 06:38:15.607774 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:15.607784 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:15.607830 (+ 46us) server_negotiation.cc:407] Connection header received
0810 06:38:15.609393 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:15.609412 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:15.609796 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:15.610149 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:15.611594 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.612881 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.614103 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.614798 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.614968 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:15.618921 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:15.618928 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:15.618942 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:15.618975 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:15.621968 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.622709 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.622716 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.622721 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.622899 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.623276 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.623280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.623283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.623782 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:15.624099 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:15.624524 (+ 425us) server_negotiation.cc:299] Negotiation successful
0810 06:38:15.624917 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":320,"threads_started":1}
I20260810 06:38:15.630690 12580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a743dbe05b4058a24dfe33b83883cc" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40097 } }
I20260810 06:38:15.631012 12580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:15.631114 12580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45a743dbe05b4058a24dfe33b83883cc, State: Initialized, Role: FOLLOWER
I20260810 06:38:15.631639 12580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a743dbe05b4058a24dfe33b83883cc" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40097 } }
I20260810 06:38:15.631913 12580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:38:15.632067 12580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:38:15.632252 12580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:15.634267 12580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a743dbe05b4058a24dfe33b83883cc" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40097 } }
I20260810 06:38:15.634774 12580 leader_election.cc:302] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45a743dbe05b4058a24dfe33b83883cc; no voters:
I20260810 06:38:15.635255 12580 leader_election.cc:288] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:38:15.635380 12592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:15.635994 12592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc [term 1 LEADER]: Becoming Leader. State: Replica: 45a743dbe05b4058a24dfe33b83883cc, State: Running, Role: LEADER
I20260810 06:38:15.636503 12592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a743dbe05b4058a24dfe33b83883cc" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40097 } }
I20260810 06:38:15.637226 12580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:15.639060 12594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45a743dbe05b4058a24dfe33b83883cc. Latest consensus state: current_term: 1 leader_uuid: "45a743dbe05b4058a24dfe33b83883cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a743dbe05b4058a24dfe33b83883cc" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40097 } } }
I20260810 06:38:15.639159 12593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45a743dbe05b4058a24dfe33b83883cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a743dbe05b4058a24dfe33b83883cc" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40097 } } }
I20260810 06:38:15.639462 12593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:15.639465 12594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:15.640281 12606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:38:15.645226 12606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:38:15.653944 12606 catalog_manager.cc:1347] Generated new cluster ID: f921dc3bb6ca4b5ab7e36d1c149a3cc5
I20260810 06:38:15.654043 12606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:38:15.699383 12606 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:38:15.701263 12606 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:38:15.709839 12606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc: Generated new TSK 0
I20260810 06:38:15.710670 12606 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.193.keytab.
Entry for principal oryx/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.193.keytab.
I20260810 06:38:15.773334 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40097
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.193.keytab
--principal=oryx/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:16.110951 12632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:16.111204 12632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:16.111297 12632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:16.118271 12632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:16.118382 12632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:16.118470 12632 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:38:16.118556 12632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:38:16.128041 12632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.193.keytab
--principal=oryx/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.59.254:40097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:16.129210 12632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:16.130587 12632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:16.143013 12638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:16.143043 12639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:16.142964 12645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:16.143802 12632 server_base.cc:1034] running on GCE node
I20260810 06:38:16.144342 12632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:16.145205 12632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:16.146430 12632 hybrid_clock.cc:648] HybridClock initialized: now 1786343896146345 us; error 39 us; skew 500 ppm
I20260810 06:38:16.146804 12632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:16 dist-test-slave-4tdj krb5kdc[12435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343896, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:16.149933 12632 init.cc:377] Logged in from keytab as oryx/127.31.59.193@KRBTEST.COM (short username oryx)
I20260810 06:38:16.151328 12632 webserver.cc:460] Webserver started at http://127.31.59.193:46673/ using document root <none> and password file <none>
I20260810 06:38:16.151985 12632 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:16.152084 12632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:16.152379 12632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:16.155170 12632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "7ca70c26167a447ebaa59ab3504897b5"
format_stamp: "Formatted at 2026-08-10 06:38:16 on dist-test-slave-4tdj"
server_key: "bd842fcdb6ca51c07e9b908719428979"
server_key_iv: "e80c8d5cc9d866b2448e706fb9554a6e"
server_key_version: "encryptionkey@0"
I20260810 06:38:16.155922 12632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "7ca70c26167a447ebaa59ab3504897b5"
format_stamp: "Formatted at 2026-08-10 06:38:16 on dist-test-slave-4tdj"
server_key: "bd842fcdb6ca51c07e9b908719428979"
server_key_iv: "e80c8d5cc9d866b2448e706fb9554a6e"
server_key_version: "encryptionkey@0"
I20260810 06:38:16.160766 12632 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:38:16.164049 12657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:16.165378 12632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:38:16.165555 12632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "7ca70c26167a447ebaa59ab3504897b5"
format_stamp: "Formatted at 2026-08-10 06:38:16 on dist-test-slave-4tdj"
server_key: "bd842fcdb6ca51c07e9b908719428979"
server_key_iv: "e80c8d5cc9d866b2448e706fb9554a6e"
server_key_version: "encryptionkey@0"
I20260810 06:38:16.165750 12632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:16.179834 12632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:16.183669 12632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:16.183908 12632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:16.186523 12632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:16.186626 12632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:16.186753 12632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:16.186821 12632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:38:16 dist-test-slave-4tdj krb5kdc[12435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343896, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM
I20260810 06:38:16.209323 12672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:16.187176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:52793 (local address 127.31.59.254:40097)
0810 06:38:16.187703 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:16.187708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:16.188704 (+ 996us) server_negotiation.cc:407] Connection header received
0810 06:38:16.190713 (+ 2009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:16.190719 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:16.190804 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:16.190899 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:16.192895 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:16.193892 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:16.196266 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:16.196679 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:16.196787 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:16.201277 (+ 4490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:16.201284 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:16.201288 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:16.201332 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:16.204727 (+ 3395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:16.205611 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:16.205617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:16.205620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:16.205773 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:16.206286 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:16.206310 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:16.206314 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:16.206677 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:16.206805 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:16.208721 (+ 1916us) server_negotiation.cc:299] Negotiation successful
0810 06:38:16.208964 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":310,"threads_started":1}
I20260810 06:38:16.209640 12673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:16.187741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40097 (local address 127.31.59.193:52793)
0810 06:38:16.188545 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:16.188588 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:16.190420 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:16.191147 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:16.191155 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:16.191825 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:16.192724 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:16.192754 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:16.194042 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:16.194047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:16.196070 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:16.196094 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:16.196921 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:16.196928 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:16.197218 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:16.197880 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:16.197913 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:16.200971 (+ 3058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:16.204874 (+ 3903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:16.204883 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:16.204903 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:16.205475 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:16.205904 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:16.205910 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:16.205913 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:16.206164 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:16.206800 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:16.206807 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:16.207157 (+ 350us) client_negotiation.cc:769] Sending connection context
0810 06:38:16.207481 (+ 324us) client_negotiation.cc:240] Negotiation successful
0810 06:38:16.207994 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":351,"threads_started":1}
I20260810 06:38:16.231050 12632 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:40419
I20260810 06:38:16.231109 12876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:40419 every 8 connection(s)
I20260810 06:38:16.231750 12672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:16.215818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:58941 (local address 127.31.59.254:40097)
0810 06:38:16.215983 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:16.215988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:16.216188 (+ 200us) server_negotiation.cc:407] Connection header received
0810 06:38:16.216333 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:16.216339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:16.216405 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:16.216479 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:16.217340 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:16.218073 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:16.219951 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:16.220364 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:16.220472 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:16.225640 (+ 5168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:16.225648 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:16.225653 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:16.225694 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:16.229023 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:16.229688 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:16.229694 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:16.229697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:16.229871 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:16.230320 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:16.230326 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:16.230330 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:16.230636 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:16.230781 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:16.231198 (+ 417us) server_negotiation.cc:299] Negotiation successful
0810 06:38:16.231410 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 06:38:16.231859 12811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:16.215385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40097 (local address 127.31.59.193:58941)
0810 06:38:16.216100 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:16.216116 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:16.216233 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:16.216515 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:16.216519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:16.216928 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:16.217210 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:16.217218 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:16.218224 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:16.218228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:16.219820 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:16.219834 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:16.220540 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:16.220546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:16.220711 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:16.224301 (+ 3590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:16.224325 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:16.225418 (+ 1093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:16.229157 (+ 3739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:16.229166 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:16.229170 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:16.229574 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:16.229990 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:16.229995 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:16.229998 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:16.230162 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:16.230753 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:16.230758 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:16.230949 (+ 191us) client_negotiation.cc:769] Sending connection context
0810 06:38:16.231085 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 06:38:16.231433 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":349,"threads_started":1}
I20260810 06:38:16.232589 12632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:38:16.234714 12877 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40097
I20260810 06:38:16.234933 12877 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:16.235589 12877 heartbeater.cc:511] Master 127.31.59.254:40097 requested a full tablet report, sending...
I20260810 06:38:16.237474 12507 ts_manager.cc:194] Registered new tserver with Master: 7ca70c26167a447ebaa59ab3504897b5 (127.31.59.193:40419)
I20260810 06:38:16.238420 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 12632
I20260810 06:38:16.238628 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:38:16.239019 12507 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:52793
I20260810 06:38:16.239245 31983 external_mini_cluster.cc:1428] Setting key 97ae05e79ce07bea54b1baad3368a353
WARNING: no policy specified for oryx/127.31.59.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.194.keytab.
Entry for principal oryx/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.194.keytab.
I20260810 06:38:16.286917 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40097
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.194.keytab
--principal=oryx/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:16.614809 12884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:16.615145 12884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:16.615295 12884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:16.623512 12884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:16.623615 12884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:16.623694 12884 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:38:16.623773 12884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:38:16.633021 12884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.194.keytab
--principal=oryx/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.59.254:40097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:16.634110 12884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:16.635377 12884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:16.647773 12891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:16.647830 12892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:16.647861 12896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:16.648567 12884 server_base.cc:1034] running on GCE node
I20260810 06:38:16.649205 12884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:16.650117 12884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:16.651358 12884 hybrid_clock.cc:648] HybridClock initialized: now 1786343896651280 us; error 41 us; skew 500 ppm
I20260810 06:38:16.651675 12884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:16 dist-test-slave-4tdj krb5kdc[12435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343896, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:16.654785 12884 init.cc:377] Logged in from keytab as oryx/127.31.59.194@KRBTEST.COM (short username oryx)
I20260810 06:38:16.656203 12884 webserver.cc:460] Webserver started at http://127.31.59.194:41931/ using document root <none> and password file <none>
I20260810 06:38:16.656930 12884 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:16.657042 12884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:16.657377 12884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:16.660331 12884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "afe31bcaaa2e4bf5a9251629035d2264"
format_stamp: "Formatted at 2026-08-10 06:38:16 on dist-test-slave-4tdj"
server_key: "fbf1980f9c5662e1608e510ec5412015"
server_key_iv: "b9cca8e34f4842e7187c69daa1a19ba6"
server_key_version: "encryptionkey@0"
I20260810 06:38:16.661108 12884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "afe31bcaaa2e4bf5a9251629035d2264"
format_stamp: "Formatted at 2026-08-10 06:38:16 on dist-test-slave-4tdj"
server_key: "fbf1980f9c5662e1608e510ec5412015"
server_key_iv: "b9cca8e34f4842e7187c69daa1a19ba6"
server_key_version: "encryptionkey@0"
I20260810 06:38:16.666121 12884 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260810 06:38:16.669669 12907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:16.671013 12884 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:38:16.671164 12884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "afe31bcaaa2e4bf5a9251629035d2264"
format_stamp: "Formatted at 2026-08-10 06:38:16 on dist-test-slave-4tdj"
server_key: "fbf1980f9c5662e1608e510ec5412015"
server_key_iv: "b9cca8e34f4842e7187c69daa1a19ba6"
server_key_version: "encryptionkey@0"
I20260810 06:38:16.671360 12884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:16.693979 12884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:16.697688 12884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:16.697926 12884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:16.700506 12884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:16.700621 12884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:16.700748 12884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:16.700817 12884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:38:16 dist-test-slave-4tdj krb5kdc[12435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343896, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM
I20260810 06:38:16.723625 12672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:16.701158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:49495 (local address 127.31.59.254:40097)
0810 06:38:16.701366 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:16.701372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:16.702787 (+ 1415us) server_negotiation.cc:407] Connection header received
0810 06:38:16.705088 (+ 2301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:16.705095 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:16.705197 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:16.705324 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:16.707841 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:16.708845 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:16.711350 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:16.711984 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:16.712127 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:16.717127 (+ 5000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:16.717132 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:16.717136 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:16.717173 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:16.720443 (+ 3270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:16.721140 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:16.721146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:16.721150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:16.721273 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:16.721702 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:16.721707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:16.721711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:16.721982 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:16.722112 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:16.722974 (+ 862us) server_negotiation.cc:299] Negotiation successful
0810 06:38:16.723242 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 06:38:16.724470 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:16.701721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40097 (local address 127.31.59.194:49495)
0810 06:38:16.702611 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:16.702658 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:16.704744 (+ 2086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:16.705764 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:16.705787 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:16.706664 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:16.707609 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:16.707641 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:16.709085 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:16.709089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:16.711141 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:16.711155 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:16.712257 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:16.712266 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:16.712564 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:16.713367 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:16.713409 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:16.716871 (+ 3462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:16.720581 (+ 3710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:16.720591 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:16.720605 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:16.721007 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:16.721376 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:16.721381 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:16.721385 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:16.721587 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:16.722111 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:16.722118 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:16.722501 (+ 383us) client_negotiation.cc:769] Sending connection context
0810 06:38:16.722813 (+ 312us) client_negotiation.cc:240] Negotiation successful
0810 06:38:16.723229 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":365,"threads_started":1}
I20260810 06:38:16.745661 13034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:16.731096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40097 (local address 127.31.59.194:48117)
0810 06:38:16.731689 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:16.731708 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:16.731855 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:16.732386 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:16.732392 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:16.733010 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:16.733395 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:16.733408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:16.734483 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:16.734490 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:16.736181 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:16.736198 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:16.737041 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:16.737048 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:16.737325 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:16.738053 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:16.738088 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:16.739222 (+ 1134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:16.743121 (+ 3899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:16.743125 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:16.743128 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:16.743541 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:16.743958 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:16.743961 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:16.743964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:16.744097 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:16.744763 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:16.744767 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:16.744918 (+ 151us) client_negotiation.cc:769] Sending connection context
0810 06:38:16.745025 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 06:38:16.745260 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":458,"threads_started":1}
I20260810 06:38:16.745719 12672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:16.731461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:48117 (local address 127.31.59.254:40097)
0810 06:38:16.731667 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:16.731674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:16.731790 (+ 116us) server_negotiation.cc:407] Connection header received
0810 06:38:16.731976 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:16.731981 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:16.732092 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:16.732199 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:16.733551 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:16.734266 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:16.736366 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:16.736832 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:16.736943 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:16.739477 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:16.739482 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:16.739485 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:16.739523 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:16.742978 (+ 3455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:16.743644 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:16.743650 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:16.743654 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:16.743831 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:16.744187 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:16.744192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:16.744195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:16.744640 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:16.744788 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:16.745133 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 06:38:16.745381 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:38:16.754523 12884 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:35001
I20260810 06:38:16.754608 13099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:35001 every 8 connection(s)
I20260810 06:38:16.755689 12884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:38:16.757551 13100 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40097
I20260810 06:38:16.757782 13100 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:16.758486 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 12884
I20260810 06:38:16.758646 13100 heartbeater.cc:511] Master 127.31.59.254:40097 requested a full tablet report, sending...
I20260810 06:38:16.758706 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:38:16.759274 31983 external_mini_cluster.cc:1428] Setting key d1dbb225b67c48cb4aa47b24ef6b0a3f
I20260810 06:38:16.760485 12507 ts_manager.cc:194] Registered new tserver with Master: afe31bcaaa2e4bf5a9251629035d2264 (127.31.59.194:35001)
I20260810 06:38:16.761475 12507 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:49495
WARNING: no policy specified for oryx/127.31.59.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.195.keytab.
Entry for principal oryx/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.195.keytab.
I20260810 06:38:16.809041 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40097
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.195.keytab
--principal=oryx/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:17.130509 13106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:17.130780 13106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:17.130882 13106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:17.137924 13106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:17.138031 13106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:17.138123 13106 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:38:17.138219 13106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:38:17.148115 13106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.195.keytab
--principal=oryx/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.59.254:40097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:17.149308 13106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:17.150648 13106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:17.163312 13115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:17.163354 13114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:17.163386 13120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:17.164201 13106 server_base.cc:1034] running on GCE node
I20260810 06:38:17.164741 13106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:17.165608 13106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:17.166816 13106 hybrid_clock.cc:648] HybridClock initialized: now 1786343897166738 us; error 47 us; skew 500 ppm
I20260810 06:38:17.167110 13106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:17 dist-test-slave-4tdj krb5kdc[12435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343897, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:17.170219 13106 init.cc:377] Logged in from keytab as oryx/127.31.59.195@KRBTEST.COM (short username oryx)
I20260810 06:38:17.171576 13106 webserver.cc:460] Webserver started at http://127.31.59.195:36789/ using document root <none> and password file <none>
I20260810 06:38:17.172258 13106 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:17.172359 13106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:17.172669 13106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:17.175655 13106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "1d6785149bb9486b983950d2accae449"
format_stamp: "Formatted at 2026-08-10 06:38:17 on dist-test-slave-4tdj"
server_key: "c5b24f4ac28962447f7d62b159943593"
server_key_iv: "4b01b915e34fd56de39ab374ac6de600"
server_key_version: "encryptionkey@0"
I20260810 06:38:17.176440 13106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "1d6785149bb9486b983950d2accae449"
format_stamp: "Formatted at 2026-08-10 06:38:17 on dist-test-slave-4tdj"
server_key: "c5b24f4ac28962447f7d62b159943593"
server_key_iv: "4b01b915e34fd56de39ab374ac6de600"
server_key_version: "encryptionkey@0"
I20260810 06:38:17.181510 13106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 06:38:17.184880 13131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:17.186189 13106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 06:38:17.186388 13106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "1d6785149bb9486b983950d2accae449"
format_stamp: "Formatted at 2026-08-10 06:38:17 on dist-test-slave-4tdj"
server_key: "c5b24f4ac28962447f7d62b159943593"
server_key_iv: "4b01b915e34fd56de39ab374ac6de600"
server_key_version: "encryptionkey@0"
I20260810 06:38:17.186583 13106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:17.198261 13106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:17.202342 13106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:17.202591 13106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:17.205231 13106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:17.205317 13106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:17.205475 13106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:17.205586 13106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:38:17 dist-test-slave-4tdj krb5kdc[12435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343897, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM
I20260810 06:38:17.227823 12672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:17.205659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:60759 (local address 127.31.59.254:40097)
0810 06:38:17.205878 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:17.205885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:17.207298 (+ 1413us) server_negotiation.cc:407] Connection header received
0810 06:38:17.209981 (+ 2683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:17.210000 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:17.210095 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:17.210212 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:17.212702 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:17.213444 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:17.216139 (+ 2695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:17.216662 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:17.216790 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:17.221407 (+ 4617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:17.221411 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:17.221415 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:17.221457 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:17.224429 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:17.225359 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:17.225365 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:17.225369 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:17.225506 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:17.225928 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:17.225933 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:17.225937 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:17.226246 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:17.226412 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:17.227223 (+ 811us) server_negotiation.cc:299] Negotiation successful
0810 06:38:17.227460 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 06:38:17.228744 13148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:17.206170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40097 (local address 127.31.59.195:60759)
0810 06:38:17.207112 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:17.207161 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:17.209675 (+ 2514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:17.210521 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:17.210544 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:17.211434 (+ 890us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:17.212497 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:17.212537 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:17.213605 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:17.213611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:17.215982 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:17.216001 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:17.217085 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:17.217090 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:17.217377 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:17.218111 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:17.218139 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:17.221188 (+ 3049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:17.224564 (+ 3376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:17.224573 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:17.224593 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:17.225239 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:17.225616 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:17.225621 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:17.225625 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:17.225817 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:17.226387 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:17.226394 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:17.226803 (+ 409us) client_negotiation.cc:769] Sending connection context
0810 06:38:17.227028 (+ 225us) client_negotiation.cc:240] Negotiation successful
0810 06:38:17.227562 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":327,"threads_started":1}
I20260810 06:38:17.241811 12877 heartbeater.cc:503] Master 127.31.59.254:40097 was elected leader, sending a full tablet report...
I20260810 06:38:17.255653 12672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:17.240518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:60593 (local address 127.31.59.254:40097)
0810 06:38:17.240691 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:17.240697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:17.241399 (+ 702us) server_negotiation.cc:407] Connection header received
0810 06:38:17.241597 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:17.241602 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:17.241706 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:17.241821 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:17.243871 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:17.244729 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:17.246918 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:17.247477 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:17.247598 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:17.249915 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:17.249919 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:17.249923 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:17.249952 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:17.253057 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:17.253693 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:17.253699 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:17.253702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:17.253827 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:17.254237 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:17.254242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:17.254245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:17.254583 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:17.254715 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:17.255124 (+ 409us) server_negotiation.cc:299] Negotiation successful
0810 06:38:17.255332 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:38:17.255754 13245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:17.240608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40097 (local address 127.31.59.195:60593)
0810 06:38:17.241263 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:17.241285 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:17.241456 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:17.241849 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:17.241853 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:17.243293 (+ 1440us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:17.243711 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:17.243725 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:17.244881 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:17.244887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:17.246781 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:17.246800 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:17.247822 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:17.247830 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:17.248056 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:17.248718 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:17.248742 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:17.249742 (+ 1000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:17.253170 (+ 3428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:17.253175 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:17.253183 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:17.253581 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:17.253936 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:17.253940 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:17.253942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:17.254079 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:17.254701 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:17.254705 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:17.254876 (+ 171us) client_negotiation.cc:769] Sending connection context
0810 06:38:17.254993 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 06:38:17.255299 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":316,"threads_started":1}
I20260810 06:38:17.266999 13106 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:42567
I20260810 06:38:17.267046 13337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:42567 every 8 connection(s)
I20260810 06:38:17.268163 13106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:38:17.269999 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 13106
I20260810 06:38:17.270203 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:38:17.270185 13342 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40097
I20260810 06:38:17.270426 13342 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:17.270745 31983 external_mini_cluster.cc:1428] Setting key ef986560e8a3486e5557489b73be1fb9
I20260810 06:38:17.271067 13342 heartbeater.cc:511] Master 127.31.59.254:40097 requested a full tablet report, sending...
I20260810 06:38:17.272375 12507 ts_manager.cc:194] Registered new tserver with Master: 1d6785149bb9486b983950d2accae449 (127.31.59.195:42567)
I20260810 06:38:17.273419 12507 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:60759
I20260810 06:38:17.273598 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 06:38:17 dist-test-slave-4tdj krb5kdc[12435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343897, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:38:17 dist-test-slave-4tdj krb5kdc[12435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:17 dist-test-slave-4tdj krb5kdc[12435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:17.763972 13100 heartbeater.cc:503] Master 127.31.59.254:40097 was elected leader, sending a full tablet report...
I20260810 06:38:18.275780 13342 heartbeater.cc:503] Master 127.31.59.254:40097 was elected leader, sending a full tablet report...
W20260810 06:38:18.401131 13352 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.080s user 0.000s sys 0.017s
W20260810 06:38:18.402454 12672 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:17.314958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56658 (local address 127.31.59.254:40097)
0810 06:38:17.315234 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:17.315241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:17.315510 (+ 269us) server_negotiation.cc:407] Connection header received
0810 06:38:17.315763 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:17.315768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:17.315843 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:17.315933 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:17.317360 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:17.318195 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:17.319587 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:17.320013 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:17.320140 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:18.402088 (+1081948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56658: BlockingRecv error: recv got EOF from 127.0.0.1:56658 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 10 06:38:18 dist-test-slave-4tdj krb5kdc[12435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343897, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM
I20260810 06:38:18.418998 12672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:18.406119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38004 (local address 127.31.59.254:40097)
0810 06:38:18.406281 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:18.406285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:18.406647 (+ 362us) server_negotiation.cc:407] Connection header received
0810 06:38:18.406762 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:18.406767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:18.406860 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:18.406951 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:18.408123 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.408866 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.410051 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.410528 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.410633 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:18.414405 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:18.414410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:18.414414 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:18.414449 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:18.416896 (+ 2447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:18.417473 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:18.417477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:18.417479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:18.417603 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:18.417898 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:18.417901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:18.417903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:18.418162 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:18.418437 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:18.418595 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 06:38:18.418758 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:38:18.423058 12507 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38004:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:38:18.430102 12507 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:38:18.467087 13372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:18.443204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42190 (local address 127.31.59.194:35001)
0810 06:38:18.455901 (+ 12697us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:18.455913 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:18.457499 (+ 1586us) server_negotiation.cc:407] Connection header received
0810 06:38:18.457856 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:18.457864 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:18.458126 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:18.458356 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:18.459351 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.461009 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.464401 (+ 3392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.465657 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.465788 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:18.466010 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:18.466614 (+ 604us) server_negotiation.cc:299] Negotiation successful
0810 06:38:18.466786 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12469,"thread_start_us":12719,"threads_started":1}
I20260810 06:38:18.467176 13370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:18.442619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46830 (local address 127.31.59.195:42567)
0810 06:38:18.457609 (+ 14990us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:18.457629 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:18.457657 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:38:18.457758 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:18.457766 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:18.458037 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:18.459206 (+ 1169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:18.459454 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.461037 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.464047 (+ 3010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.465536 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.465665 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:18.465883 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:18.466738 (+ 855us) server_negotiation.cc:299] Negotiation successful
0810 06:38:18.466891 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14826,"thread_start_us":14684,"threads_started":1}
I20260810 06:38:18.467191 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:18.442412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:42567 (local address 127.0.0.1:46830)
0810 06:38:18.456762 (+ 14350us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:18.456804 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:18.457013 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:18.458286 (+ 1273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:18.458346 (+ 60us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:18.458391 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:18.458959 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:18.458972 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.461200 (+ 2228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.461206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:18.463887 (+ 2681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:18.463901 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.466021 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.466028 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:18.466246 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:18.466437 (+ 191us) client_negotiation.cc:769] Sending connection context
0810 06:38:18.466545 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 06:38:18.466766 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14058,"mutex_wait_us":79,"thread_start_us":15100,"threads_started":1}
I20260810 06:38:18.468036 13369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:18.442800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:35001 (local address 127.0.0.1:42190)
0810 06:38:18.457363 (+ 14563us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:18.457382 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:18.457632 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:18.458612 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:18.458618 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:18.458646 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:18.459157 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:18.459168 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.461239 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.461244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:18.464260 (+ 3016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:18.464270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.466150 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.466154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:18.466376 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:18.466425 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 06:38:18.466535 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 06:38:18.467678 (+ 1143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14422,"mutex_wait_us":996,"thread_start_us":13509,"threads_started":1}
I20260810 06:38:18.470360 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:18.457988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:40419 (local address 127.0.0.1:43838)
0810 06:38:18.462325 (+ 4337us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:18.462342 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:18.462546 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:18.463133 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:18.463137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:18.463161 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:18.463563 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:18.463572 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.465526 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.465531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:18.467605 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:18.467619 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.469701 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.469707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:18.469873 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:18.469905 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:38:18.469990 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 06:38:18.470105 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4210,"thread_start_us":4102,"threads_started":1}
I20260810 06:38:18.470999 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:18.458101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43838 (local address 127.31.59.193:40419)
0810 06:38:18.461939 (+ 3838us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:18.461945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:18.462443 (+ 498us) server_negotiation.cc:407] Connection header received
0810 06:38:18.462770 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:18.462776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:18.462956 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:18.463125 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:18.463724 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.465361 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.467784 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.469237 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.469376 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:18.469563 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:18.470671 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0810 06:38:18.470799 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3700,"thread_start_us":3555,"threads_started":1}
I20260810 06:38:18.471359 13216 tablet_service.cc:1467] Processing CreateTablet for tablet 985b8a602fd143249f1c260e701909f2 (DEFAULT_TABLE table=test-table [id=512285328bb34d0694b4d69292f7db3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:18.471699 12983 tablet_service.cc:1467] Processing CreateTablet for tablet 985b8a602fd143249f1c260e701909f2 (DEFAULT_TABLE table=test-table [id=512285328bb34d0694b4d69292f7db3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:18.472919 13216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 985b8a602fd143249f1c260e701909f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:18.473039 12983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 985b8a602fd143249f1c260e701909f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:18.474843 12751 tablet_service.cc:1467] Processing CreateTablet for tablet 985b8a602fd143249f1c260e701909f2 (DEFAULT_TABLE table=test-table [id=512285328bb34d0694b4d69292f7db3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:18.476147 12751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 985b8a602fd143249f1c260e701909f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:18.488601 13380 tablet_bootstrap.cc:492] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449: Bootstrap starting.
I20260810 06:38:18.490259 13382 tablet_bootstrap.cc:492] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264: Bootstrap starting.
I20260810 06:38:18.492714 13380 tablet_bootstrap.cc:654] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:18.493031 13382 tablet_bootstrap.cc:654] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:18.494038 13382 log.cc:824] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:18.494105 13380 log.cc:824] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:18.495846 13378 tablet_bootstrap.cc:492] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5: Bootstrap starting.
I20260810 06:38:18.498744 13378 tablet_bootstrap.cc:654] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:18.500054 13378 log.cc:824] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:18.501327 13380 tablet_bootstrap.cc:492] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449: No bootstrap required, opened a new log
I20260810 06:38:18.501624 13380 ts_tablet_manager.cc:1397] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260810 06:38:18.502545 13382 tablet_bootstrap.cc:492] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264: No bootstrap required, opened a new log
I20260810 06:38:18.502840 13382 ts_tablet_manager.cc:1397] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.005s
I20260810 06:38:18.502956 13378 tablet_bootstrap.cc:492] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5: No bootstrap required, opened a new log
I20260810 06:38:18.503211 13378 ts_tablet_manager.cc:1397] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 06:38:18.510113 13382 raft_consensus.cc:348] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } }
I20260810 06:38:18.510494 13382 raft_consensus.cc:374] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:18.510603 13382 raft_consensus.cc:729] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afe31bcaaa2e4bf5a9251629035d2264, State: Initialized, Role: FOLLOWER
I20260810 06:38:18.511184 13382 consensus_queue.cc:260] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } }
I20260810 06:38:18.512153 13380 raft_consensus.cc:348] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } }
I20260810 06:38:18.512566 13380 raft_consensus.cc:374] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:18.512669 13380 raft_consensus.cc:729] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d6785149bb9486b983950d2accae449, State: Initialized, Role: FOLLOWER
I20260810 06:38:18.513265 13380 consensus_queue.cc:260] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } }
I20260810 06:38:18.513249 13378 raft_consensus.cc:348] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } }
I20260810 06:38:18.513664 13378 raft_consensus.cc:374] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:18.513800 13378 raft_consensus.cc:729] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca70c26167a447ebaa59ab3504897b5, State: Initialized, Role: FOLLOWER
I20260810 06:38:18.514478 13378 consensus_queue.cc:260] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } }
I20260810 06:38:18.516585 13378 ts_tablet_manager.cc:1428] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260810 06:38:18.518245 13380 ts_tablet_manager.cc:1428] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449: Time spent starting tablet: real 0.016s user 0.013s sys 0.003s
I20260810 06:38:18.519622 13382 ts_tablet_manager.cc:1428] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
W20260810 06:38:18.520860 13343 tablet.cc:2369] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:38:18.735975 12878 tablet.cc:2369] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:38:18.758548 13101 tablet.cc:2369] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:38:18.939502 13389 raft_consensus.cc:482] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:18.939854 13389 raft_consensus.cc:504] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } }
I20260810 06:38:18.941880 13389 leader_election.cc:288] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d6785149bb9486b983950d2accae449 (127.31.59.195:42567), 7ca70c26167a447ebaa59ab3504897b5 (127.31.59.193:40419)
I20260810 06:38:18.948770 13370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:18.942322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:48441 (local address 127.31.59.195:42567)
0810 06:38:18.942508 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:18.942515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:18.943055 (+ 540us) server_negotiation.cc:407] Connection header received
0810 06:38:18.943256 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:18.943261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:18.943363 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:18.943468 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:18.944158 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.944891 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.946936 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.947761 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.947886 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:18.947961 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:18.948391 (+ 430us) server_negotiation.cc:299] Negotiation successful
0810 06:38:18.948530 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 06:38:18.948778 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:18.942568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:60355 (local address 127.31.59.193:40419)
0810 06:38:18.942775 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:18.942781 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:18.943282 (+ 501us) server_negotiation.cc:407] Connection header received
0810 06:38:18.943400 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:18.943404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:18.943530 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:18.943633 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:18.944297 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.945034 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.947012 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.947815 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.947975 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:18.948046 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:18.948388 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 06:38:18.948522 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 06:38:18.948983 13392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:18.942102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:42567 (local address 127.31.59.194:48441)
0810 06:38:18.942916 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:18.942939 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:18.943118 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:18.943514 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:18.943519 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:18.943549 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:18.943992 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:18.944006 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.945035 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.945041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:18.946764 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:18.946780 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.947990 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.947997 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:18.948219 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:18.948265 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 06:38:18.948347 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 06:38:18.948526 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":146,"spinlock_wait_cycles":896,"thread_start_us":395,"threads_started":1}
I20260810 06:38:18.949030 13393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:18.942191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:40419 (local address 127.31.59.194:60355)
0810 06:38:18.943179 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:18.943195 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:18.943300 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:18.943666 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:18.943671 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:18.943748 (+ 77us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:18.944136 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:18.944148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.945172 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.945176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:18.946812 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:18.946825 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:18.948046 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:18.948051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:18.948186 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:18.948260 (+ 74us) client_negotiation.cc:769] Sending connection context
0810 06:38:18.948342 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 06:38:18.948523 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"mutex_wait_us":94,"spinlock_wait_cycles":32000,"thread_start_us":668,"threads_started":1}
I20260810 06:38:18.950266 12779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "985b8a602fd143249f1c260e701909f2" candidate_uuid: "afe31bcaaa2e4bf5a9251629035d2264" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca70c26167a447ebaa59ab3504897b5" is_pre_election: true
I20260810 06:38:18.950266 13256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "985b8a602fd143249f1c260e701909f2" candidate_uuid: "afe31bcaaa2e4bf5a9251629035d2264" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d6785149bb9486b983950d2accae449" is_pre_election: true
I20260810 06:38:18.950917 12779 raft_consensus.cc:2454] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afe31bcaaa2e4bf5a9251629035d2264 in term 0.
I20260810 06:38:18.950917 13256 raft_consensus.cc:2454] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afe31bcaaa2e4bf5a9251629035d2264 in term 0.
I20260810 06:38:18.951656 12908 leader_election.cc:302] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d6785149bb9486b983950d2accae449, afe31bcaaa2e4bf5a9251629035d2264; no voters:
I20260810 06:38:18.952068 13389 raft_consensus.cc:2781] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:38:18.952229 13389 raft_consensus.cc:482] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:38:18.952327 13389 raft_consensus.cc:3037] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:18.954557 13389 raft_consensus.cc:504] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } }
I20260810 06:38:18.955132 13389 leader_election.cc:288] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [CANDIDATE]: Term 1 election: Requested vote from peers 1d6785149bb9486b983950d2accae449 (127.31.59.195:42567), 7ca70c26167a447ebaa59ab3504897b5 (127.31.59.193:40419)
I20260810 06:38:18.955595 13256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "985b8a602fd143249f1c260e701909f2" candidate_uuid: "afe31bcaaa2e4bf5a9251629035d2264" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d6785149bb9486b983950d2accae449"
I20260810 06:38:18.955605 12779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "985b8a602fd143249f1c260e701909f2" candidate_uuid: "afe31bcaaa2e4bf5a9251629035d2264" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca70c26167a447ebaa59ab3504897b5"
I20260810 06:38:18.955778 12779 raft_consensus.cc:3037] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:18.955782 13256 raft_consensus.cc:3037] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:18.957926 12779 raft_consensus.cc:2454] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afe31bcaaa2e4bf5a9251629035d2264 in term 1.
I20260810 06:38:18.957942 13256 raft_consensus.cc:2454] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afe31bcaaa2e4bf5a9251629035d2264 in term 1.
I20260810 06:38:18.958333 12914 leader_election.cc:302] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ca70c26167a447ebaa59ab3504897b5, afe31bcaaa2e4bf5a9251629035d2264; no voters:
I20260810 06:38:18.958638 13389 raft_consensus.cc:2781] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:18.959040 13389 raft_consensus.cc:686] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [term 1 LEADER]: Becoming Leader. State: Replica: afe31bcaaa2e4bf5a9251629035d2264, State: Running, Role: LEADER
I20260810 06:38:18.959493 13389 consensus_queue.cc:237] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } }
I20260810 06:38:18.963856 12507 catalog_manager.cc:5495] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 reported cstate change: term changed from 0 to 1, leader changed from <none> to afe31bcaaa2e4bf5a9251629035d2264 (127.31.59.194). New cstate: current_term: 1 leader_uuid: "afe31bcaaa2e4bf5a9251629035d2264" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260810 06:38:19.041766 13409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.035520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42194 (local address 127.31.59.194:35001)
0810 06:38:19.036078 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.036084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.036107 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:38:19.036387 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.036392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.036486 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.036651 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:19.037136 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.037926 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.039455 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.039897 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.040011 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.040465 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:19.041082 (+ 617us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:38:19.041238 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.041339 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.041437 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":291,"threads_started":1}
I20260810 06:38:19.051157 13256 raft_consensus.cc:1261] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449 [term 1 FOLLOWER]: Refusing update from remote peer afe31bcaaa2e4bf5a9251629035d2264: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:19.051158 12779 raft_consensus.cc:1261] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5 [term 1 FOLLOWER]: Refusing update from remote peer afe31bcaaa2e4bf5a9251629035d2264: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:19.052698 13389 consensus_queue.cc:1035] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:19.053188 13395 consensus_queue.cc:1035] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:19.081867 12505 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:19.082171 12505 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:19.085919 12505 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc: Sending DeleteTablet for 3 replicas of tablet 985b8a602fd143249f1c260e701909f2
I20260810 06:38:19.086936 13216 tablet_service.cc:1514] Processing DeleteTablet for tablet 985b8a602fd143249f1c260e701909f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:19 UTC) from {username='oryx', principal='oryx/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:46830
I20260810 06:38:19.087121 12983 tablet_service.cc:1514] Processing DeleteTablet for tablet 985b8a602fd143249f1c260e701909f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:19 UTC) from {username='oryx', principal='oryx/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:42190
I20260810 06:38:19.087203 12751 tablet_service.cc:1514] Processing DeleteTablet for tablet 985b8a602fd143249f1c260e701909f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:19 UTC) from {username='oryx', principal='oryx/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:43838
I20260810 06:38:19.091917 12505 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38004:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:38:19.094774 12505 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:38:19.097038 13430 tablet_replica.cc:330] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5: stopping tablet replica
I20260810 06:38:19.097502 13430 raft_consensus.cc:2229] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:19.098003 13430 raft_consensus.cc:2258] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:19.099725 13427 tablet_replica.cc:330] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449: stopping tablet replica
I20260810 06:38:19.100299 13427 raft_consensus.cc:2229] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:19.100842 13427 raft_consensus.cc:2258] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:19.101539 13430 ts_tablet_manager.cc:1905] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:19.102839 13427 ts_tablet_manager.cc:1905] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:19.104055 13216 tablet_service.cc:1467] Processing CreateTablet for tablet e61dd059135d409693b2df07ed1fc6f1 (DEFAULT_TABLE table=test-table [id=c17c27f9b86346709ab067378e188008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:19.104179 12982 tablet_service.cc:1467] Processing CreateTablet for tablet e61dd059135d409693b2df07ed1fc6f1 (DEFAULT_TABLE table=test-table [id=c17c27f9b86346709ab067378e188008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:19.104599 13216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e61dd059135d409693b2df07ed1fc6f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:19.104786 12982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e61dd059135d409693b2df07ed1fc6f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:19.108099 12751 tablet_service.cc:1467] Processing CreateTablet for tablet e61dd059135d409693b2df07ed1fc6f1 (DEFAULT_TABLE table=test-table [id=c17c27f9b86346709ab067378e188008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:19.108536 12751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e61dd059135d409693b2df07ed1fc6f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:19.110460 13427 ts_tablet_manager.cc:1918] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:19.110663 13427 log.cc:1196] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/985b8a602fd143249f1c260e701909f2
I20260810 06:38:19.111294 13427 ts_tablet_manager.cc:1939] T 985b8a602fd143249f1c260e701909f2 P 1d6785149bb9486b983950d2accae449: Deleting consensus metadata
I20260810 06:38:19.113536 12481 catalog_manager.cc:4841] TS 1d6785149bb9486b983950d2accae449 (127.31.59.195:42567): tablet 985b8a602fd143249f1c260e701909f2 (table test-table [id=512285328bb34d0694b4d69292f7db3c]) successfully deleted
I20260810 06:38:19.114790 13431 tablet_bootstrap.cc:492] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264: Bootstrap starting.
I20260810 06:38:19.116803 13431 tablet_bootstrap.cc:654] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:19.117311 13433 tablet_bootstrap.cc:492] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449: Bootstrap starting.
I20260810 06:38:19.120012 13433 tablet_bootstrap.cc:654] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:19.121083 13430 ts_tablet_manager.cc:1918] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:19.122555 13429 tablet_replica.cc:330] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264: stopping tablet replica
I20260810 06:38:19.123128 13429 raft_consensus.cc:2229] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:38:19.124389 13429 raft_consensus.cc:2258] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:19.126456 13429 ts_tablet_manager.cc:1905] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:19.123733 13438 tablet_bootstrap.cc:492] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5: Bootstrap starting.
I20260810 06:38:19.128160 13433 tablet_bootstrap.cc:492] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449: No bootstrap required, opened a new log
I20260810 06:38:19.128249 13430 log.cc:1196] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/985b8a602fd143249f1c260e701909f2
I20260810 06:38:19.128420 13433 ts_tablet_manager.cc:1397] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260810 06:38:19.128788 13430 ts_tablet_manager.cc:1939] T 985b8a602fd143249f1c260e701909f2 P 7ca70c26167a447ebaa59ab3504897b5: Deleting consensus metadata
I20260810 06:38:19.129194 13433 raft_consensus.cc:348] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.129428 13438 tablet_bootstrap.cc:654] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:19.129438 13433 raft_consensus.cc:374] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:19.129534 13433 raft_consensus.cc:729] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d6785149bb9486b983950d2accae449, State: Initialized, Role: FOLLOWER
I20260810 06:38:19.129788 13433 consensus_queue.cc:260] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.130532 13433 ts_tablet_manager.cc:1428] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:38:19.131666 12486 catalog_manager.cc:4841] TS 7ca70c26167a447ebaa59ab3504897b5 (127.31.59.193:40419): tablet 985b8a602fd143249f1c260e701909f2 (table test-table [id=512285328bb34d0694b4d69292f7db3c]) successfully deleted
I20260810 06:38:19.132197 13438 tablet_bootstrap.cc:492] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5: No bootstrap required, opened a new log
I20260810 06:38:19.132292 13438 ts_tablet_manager.cc:1397] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260810 06:38:19.132826 13438 raft_consensus.cc:348] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.133121 13438 raft_consensus.cc:374] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:19.133262 13438 raft_consensus.cc:729] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca70c26167a447ebaa59ab3504897b5, State: Initialized, Role: FOLLOWER
I20260810 06:38:19.133380 13431 tablet_bootstrap.cc:492] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264: No bootstrap required, opened a new log
I20260810 06:38:19.133549 13431 ts_tablet_manager.cc:1397] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.002s
I20260810 06:38:19.133934 13438 consensus_queue.cc:260] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.134503 13431 raft_consensus.cc:348] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.134760 13431 raft_consensus.cc:374] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:19.134874 13431 raft_consensus.cc:729] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afe31bcaaa2e4bf5a9251629035d2264, State: Initialized, Role: FOLLOWER
I20260810 06:38:19.135108 13431 consensus_queue.cc:260] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.135620 13438 ts_tablet_manager.cc:1428] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260810 06:38:19.135686 13431 ts_tablet_manager.cc:1428] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:38:19.140125 13429 ts_tablet_manager.cc:1918] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:19.140286 13429 log.cc:1196] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/985b8a602fd143249f1c260e701909f2
I20260810 06:38:19.140828 13429 ts_tablet_manager.cc:1939] T 985b8a602fd143249f1c260e701909f2 P afe31bcaaa2e4bf5a9251629035d2264: Deleting consensus metadata
I20260810 06:38:19.142480 12486 catalog_manager.cc:4841] TS afe31bcaaa2e4bf5a9251629035d2264 (127.31.59.194:35001): tablet 985b8a602fd143249f1c260e701909f2 (table test-table [id=512285328bb34d0694b4d69292f7db3c]) successfully deleted
I20260810 06:38:19.174418 13413 raft_consensus.cc:482] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:19.174716 13413 raft_consensus.cc:504] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.176324 13414 raft_consensus.cc:482] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:19.176681 13413 leader_election.cc:288] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers afe31bcaaa2e4bf5a9251629035d2264 (127.31.59.194:35001), 1d6785149bb9486b983950d2accae449 (127.31.59.195:42567)
I20260810 06:38:19.176578 13414 raft_consensus.cc:504] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.178254 13414 leader_election.cc:288] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ca70c26167a447ebaa59ab3504897b5 (127.31.59.193:40419), afe31bcaaa2e4bf5a9251629035d2264 (127.31.59.194:35001)
I20260810 06:38:19.195359 13409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.177938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:57723 (local address 127.31.59.194:35001)
0810 06:38:19.178128 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.178135 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.187458 (+ 9323us) server_negotiation.cc:407] Connection header received
0810 06:38:19.187929 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.187935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.188017 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.188084 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:19.188999 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.190185 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.192866 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.194152 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.194325 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.194401 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.194938 (+ 537us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.195072 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:38:19.196123 13447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.176880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:35001 (local address 127.31.59.193:57723)
0810 06:38:19.187344 (+ 10464us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:19.187367 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:19.187752 (+ 385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:19.188326 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:19.188330 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:19.188361 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:19.188797 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.188812 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.190435 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.190440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.192699 (+ 2259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.192712 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.194577 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.194583 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.194753 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.194788 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 06:38:19.195628 (+ 840us) client_negotiation.cc:240] Negotiation successful
0810 06:38:19.195767 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10323,"thread_start_us":12928,"threads_started":1}
I20260810 06:38:19.196319 13420 raft_consensus.cc:482] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:19.196547 13420 raft_consensus.cc:504] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.197449 13420 leader_election.cc:288] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ca70c26167a447ebaa59ab3504897b5 (127.31.59.193:40419), 1d6785149bb9486b983950d2accae449 (127.31.59.195:42567)
I20260810 06:38:19.197297 13024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61dd059135d409693b2df07ed1fc6f1" candidate_uuid: "7ca70c26167a447ebaa59ab3504897b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afe31bcaaa2e4bf5a9251629035d2264" is_pre_election: true
I20260810 06:38:19.197767 13370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.177660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:55815 (local address 127.31.59.195:42567)
0810 06:38:19.178450 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.178456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.189800 (+ 11344us) server_negotiation.cc:407] Connection header received
0810 06:38:19.190193 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.190199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.190278 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.190358 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:19.191236 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.192305 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.194611 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.195775 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.195903 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.195970 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.197372 (+ 1402us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.197502 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668}
I20260810 06:38:19.197772 13024 raft_consensus.cc:2454] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ca70c26167a447ebaa59ab3504897b5 in term 0.
I20260810 06:38:19.198345 12779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61dd059135d409693b2df07ed1fc6f1" candidate_uuid: "afe31bcaaa2e4bf5a9251629035d2264" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca70c26167a447ebaa59ab3504897b5" is_pre_election: true
I20260810 06:38:19.198523 13444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.177262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:42567 (local address 127.31.59.193:55815)
0810 06:38:19.189707 (+ 12445us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:19.189724 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:19.190030 (+ 306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:19.190545 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:19.190549 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:19.190583 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:19.191023 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.191039 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.192466 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.192472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.194442 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.194457 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.197016 (+ 2559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.197021 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.197188 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.197219 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:38:19.197962 (+ 743us) client_negotiation.cc:240] Negotiation successful
0810 06:38:19.198088 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12329,"mutex_wait_us":1,"thread_start_us":10213,"threads_started":1}
I20260810 06:38:19.198614 12779 raft_consensus.cc:2454] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afe31bcaaa2e4bf5a9251629035d2264 in term 0.
I20260810 06:38:19.198978 13256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61dd059135d409693b2df07ed1fc6f1" candidate_uuid: "afe31bcaaa2e4bf5a9251629035d2264" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d6785149bb9486b983950d2accae449" is_pre_election: true
I20260810 06:38:19.199070 12667 leader_election.cc:302] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ca70c26167a447ebaa59ab3504897b5, afe31bcaaa2e4bf5a9251629035d2264; no voters:
I20260810 06:38:19.199065 13253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61dd059135d409693b2df07ed1fc6f1" candidate_uuid: "7ca70c26167a447ebaa59ab3504897b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d6785149bb9486b983950d2accae449" is_pre_election: true
I20260810 06:38:19.199250 13256 raft_consensus.cc:2454] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afe31bcaaa2e4bf5a9251629035d2264 in term 0.
I20260810 06:38:19.199474 13413 raft_consensus.cc:2781] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:38:19.199569 12914 leader_election.cc:302] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ca70c26167a447ebaa59ab3504897b5, afe31bcaaa2e4bf5a9251629035d2264; no voters:
I20260810 06:38:19.199595 13413 raft_consensus.cc:482] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:38:19.199681 13413 raft_consensus.cc:3037] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:19.200541 13420 raft_consensus.cc:2781] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:38:19.200651 13420 raft_consensus.cc:482] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:38:19.200739 13420 raft_consensus.cc:3037] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:19.202819 13413 raft_consensus.cc:504] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.203475 13413 leader_election.cc:288] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [CANDIDATE]: Term 1 election: Requested vote from peers afe31bcaaa2e4bf5a9251629035d2264 (127.31.59.194:35001), 1d6785149bb9486b983950d2accae449 (127.31.59.195:42567)
I20260810 06:38:19.204599 13253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61dd059135d409693b2df07ed1fc6f1" candidate_uuid: "7ca70c26167a447ebaa59ab3504897b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d6785149bb9486b983950d2accae449"
I20260810 06:38:19.204831 13253 raft_consensus.cc:3037] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:19.205097 13420 raft_consensus.cc:504] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.205787 13420 leader_election.cc:288] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [CANDIDATE]: Term 1 election: Requested vote from peers 7ca70c26167a447ebaa59ab3504897b5 (127.31.59.193:40419), 1d6785149bb9486b983950d2accae449 (127.31.59.195:42567)
I20260810 06:38:19.206902 13024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61dd059135d409693b2df07ed1fc6f1" candidate_uuid: "7ca70c26167a447ebaa59ab3504897b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afe31bcaaa2e4bf5a9251629035d2264"
I20260810 06:38:19.207165 13024 raft_consensus.cc:2379] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7ca70c26167a447ebaa59ab3504897b5 in current term 1: Already voted for candidate afe31bcaaa2e4bf5a9251629035d2264 in this term.
I20260810 06:38:19.207098 13256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61dd059135d409693b2df07ed1fc6f1" candidate_uuid: "afe31bcaaa2e4bf5a9251629035d2264" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d6785149bb9486b983950d2accae449"
I20260810 06:38:19.208125 12779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61dd059135d409693b2df07ed1fc6f1" candidate_uuid: "afe31bcaaa2e4bf5a9251629035d2264" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca70c26167a447ebaa59ab3504897b5"
I20260810 06:38:19.208400 12779 raft_consensus.cc:2379] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate afe31bcaaa2e4bf5a9251629035d2264 in current term 1: Already voted for candidate 7ca70c26167a447ebaa59ab3504897b5 in this term.
I20260810 06:38:19.208810 12914 leader_election.cc:302] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: afe31bcaaa2e4bf5a9251629035d2264; no voters: 1d6785149bb9486b983950d2accae449, 7ca70c26167a447ebaa59ab3504897b5
I20260810 06:38:19.209419 13253 raft_consensus.cc:2454] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ca70c26167a447ebaa59ab3504897b5 in term 1.
I20260810 06:38:19.209985 12658 leader_election.cc:302] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1d6785149bb9486b983950d2accae449, 7ca70c26167a447ebaa59ab3504897b5; no voters: afe31bcaaa2e4bf5a9251629035d2264
I20260810 06:38:19.210485 13413 raft_consensus.cc:2781] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:19.210947 13420 raft_consensus.cc:2726] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 06:38:19.211509 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.179249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:48239 (local address 127.31.59.193:40419)
0810 06:38:19.179415 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.179420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.190471 (+ 11051us) server_negotiation.cc:407] Connection header received
0810 06:38:19.199992 (+ 9521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.199998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.200066 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.200124 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:19.201395 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.202579 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.208997 (+ 6418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.210253 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.210421 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.210506 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.211098 (+ 592us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.211229 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:38:19.212359 13448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.178780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:40419 (local address 127.31.59.195:48239)
0810 06:38:19.190372 (+ 11592us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:19.190391 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:19.199816 (+ 9425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:19.200320 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:19.200325 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:19.200350 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:19.200670 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.200681 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.202772 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.202777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.206158 (+ 3381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.206174 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.210668 (+ 4494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.210674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.210845 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.210946 (+ 101us) client_negotiation.cc:769] Sending connection context
0810 06:38:19.211757 (+ 811us) client_negotiation.cc:240] Negotiation successful
0810 06:38:19.211909 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11445,"thread_start_us":11722,"threads_started":1}
I20260810 06:38:19.212720 13413 raft_consensus.cc:686] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [term 1 LEADER]: Becoming Leader. State: Replica: 7ca70c26167a447ebaa59ab3504897b5, State: Running, Role: LEADER
I20260810 06:38:19.213193 13413 consensus_queue.cc:237] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.213642 12779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61dd059135d409693b2df07ed1fc6f1" candidate_uuid: "1d6785149bb9486b983950d2accae449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca70c26167a447ebaa59ab3504897b5" is_pre_election: true
I20260810 06:38:19.215556 13409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.190950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:52397 (local address 127.31.59.194:35001)
0810 06:38:19.195583 (+ 4633us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.195589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.208782 (+ 13193us) server_negotiation.cc:407] Connection header received
0810 06:38:19.208973 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.208978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.209075 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.209162 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:19.209921 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.210925 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.213367 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.214525 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.214664 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.214740 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.215113 (+ 373us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.215243 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4525,"thread_start_us":5577,"threads_started":1}
I20260810 06:38:19.216233 13451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.199565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:35001 (local address 127.31.59.195:52397)
0810 06:38:19.208687 (+ 9122us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:19.208706 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:19.208833 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:19.209313 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:19.209317 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:19.209341 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:19.209756 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.209769 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.211112 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.211117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.213178 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.213193 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.214727 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.214734 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.214915 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.214951 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:38:19.215788 (+ 837us) client_negotiation.cc:240] Negotiation successful
0810 06:38:19.215920 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9001,"thread_start_us":8808,"threads_started":1}
I20260810 06:38:19.216879 13024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61dd059135d409693b2df07ed1fc6f1" candidate_uuid: "1d6785149bb9486b983950d2accae449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afe31bcaaa2e4bf5a9251629035d2264" is_pre_election: true
I20260810 06:38:19.217154 13024 raft_consensus.cc:2379] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1d6785149bb9486b983950d2accae449 in current term 1: Already voted for candidate afe31bcaaa2e4bf5a9251629035d2264 in this term.
I20260810 06:38:19.217221 12505 catalog_manager.cc:5495] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ca70c26167a447ebaa59ab3504897b5 (127.31.59.193). New cstate: current_term: 1 leader_uuid: "7ca70c26167a447ebaa59ab3504897b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } health_report { overall_health: UNKNOWN } } }
I20260810 06:38:19.217584 13142 leader_election.cc:302] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1d6785149bb9486b983950d2accae449; no voters: 7ca70c26167a447ebaa59ab3504897b5, afe31bcaaa2e4bf5a9251629035d2264
I20260810 06:38:19.217991 13414 raft_consensus.cc:2726] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 10 06:38:19 dist-test-slave-4tdj krb5kdc[12435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343895, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.254@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM
I20260810 06:38:19.271000 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.256686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40097 (local address 127.0.0.1:38020)
0810 06:38:19.257368 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:19.257392 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:19.257545 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:19.257953 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:19.257958 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:19.258673 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:19.258919 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.258928 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.260013 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.260016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.260937 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.260949 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.261660 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.261665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.261821 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.262438 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:19.262456 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:19.264567 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:19.267934 (+ 3367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:19.267947 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:19.267956 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:19.268607 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:19.268998 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:19.269003 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:19.269007 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:19.269212 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:19.269823 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:19.269831 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:19.270039 (+ 208us) client_negotiation.cc:769] Sending connection context
0810 06:38:19.270228 (+ 189us) client_negotiation.cc:240] Negotiation successful
0810 06:38:19.270566 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":354,"threads_started":1}
I20260810 06:38:19.271087 13468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.256728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38020 (local address 127.31.59.254:40097)
0810 06:38:19.257700 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.257704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.257721 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:38:19.257759 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.257762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.257814 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.257899 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:19.259076 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.259882 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.261087 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.261499 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.261615 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.264754 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:19.264759 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:19.264762 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:19.264790 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:19.267790 (+ 3000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:19.268720 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:19.268725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:19.268728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:19.268878 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:19.269309 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:19.269314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:19.269317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:19.269695 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:19.269824 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.270419 (+ 595us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.270711 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":839,"thread_start_us":638,"threads_started":1}
I20260810 06:38:19.277163 12505 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:38020:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 06:38:19.278406 12505 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:38:19.285883 12983 tablet_service.cc:1467] Processing CreateTablet for tablet a0489d81e52e4b1491d3ce597aa1a340 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=87505e6673f94a009ed24b44a08794b6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:38:19.286144 12751 tablet_service.cc:1467] Processing CreateTablet for tablet a0489d81e52e4b1491d3ce597aa1a340 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=87505e6673f94a009ed24b44a08794b6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:38:19.286516 12983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0489d81e52e4b1491d3ce597aa1a340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:19.286726 12751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0489d81e52e4b1491d3ce597aa1a340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:19.287361 13216 tablet_service.cc:1467] Processing CreateTablet for tablet a0489d81e52e4b1491d3ce597aa1a340 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=87505e6673f94a009ed24b44a08794b6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:38:19.287828 13216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0489d81e52e4b1491d3ce597aa1a340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:19.294503 13438 tablet_bootstrap.cc:492] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5: Bootstrap starting.
I20260810 06:38:19.294898 13431 tablet_bootstrap.cc:492] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264: Bootstrap starting.
I20260810 06:38:19.296170 13438 tablet_bootstrap.cc:654] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:19.296350 13431 tablet_bootstrap.cc:654] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:19.297003 13433 tablet_bootstrap.cc:492] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449: Bootstrap starting.
I20260810 06:38:19.298568 13431 tablet_bootstrap.cc:492] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264: No bootstrap required, opened a new log
I20260810 06:38:19.298692 13433 tablet_bootstrap.cc:654] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:19.298780 13431 ts_tablet_manager.cc:1397] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 06:38:19.299458 13431 raft_consensus.cc:348] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.299820 13431 raft_consensus.cc:374] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:19.299952 13431 raft_consensus.cc:729] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afe31bcaaa2e4bf5a9251629035d2264, State: Initialized, Role: FOLLOWER
I20260810 06:38:19.300354 13431 consensus_queue.cc:260] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.300530 13433 tablet_bootstrap.cc:492] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449: No bootstrap required, opened a new log
I20260810 06:38:19.300668 13433 ts_tablet_manager.cc:1397] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 06:38:19.300776 13420 tablet_replica.cc:439] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } }
I20260810 06:38:19.301065 13420 tablet_replica.cc:442] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:38:19.301441 13431 ts_tablet_manager.cc:1428] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:38:19.301368 13433 raft_consensus.cc:348] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.301613 13433 raft_consensus.cc:374] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:19.301692 13433 raft_consensus.cc:729] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d6785149bb9486b983950d2accae449, State: Initialized, Role: FOLLOWER
I20260810 06:38:19.302088 13433 consensus_queue.cc:260] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.302611 13414 tablet_replica.cc:439] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } }
I20260810 06:38:19.302852 13414 tablet_replica.cc:442] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:38:19.303284 13433 ts_tablet_manager.cc:1428] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:38:19.304065 13438 tablet_bootstrap.cc:492] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5: No bootstrap required, opened a new log
I20260810 06:38:19.304231 13438 ts_tablet_manager.cc:1397] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.001s
I20260810 06:38:19.305061 13438 raft_consensus.cc:348] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.305347 13438 raft_consensus.cc:374] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:19.305434 13438 raft_consensus.cc:729] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca70c26167a447ebaa59ab3504897b5, State: Initialized, Role: FOLLOWER
I20260810 06:38:19.305652 13438 consensus_queue.cc:260] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.306151 13438 ts_tablet_manager.cc:1428] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 06:38:19.306212 13413 tablet_replica.cc:439] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } }
I20260810 06:38:19.306512 13413 tablet_replica.cc:442] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:38:19.682600 13420 raft_consensus.cc:482] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:19.682843 13420 raft_consensus.cc:504] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.683584 13420 leader_election.cc:288] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ca70c26167a447ebaa59ab3504897b5 (127.31.59.193:40419), 1d6785149bb9486b983950d2accae449 (127.31.59.195:42567)
I20260810 06:38:19.684192 12779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0489d81e52e4b1491d3ce597aa1a340" candidate_uuid: "afe31bcaaa2e4bf5a9251629035d2264" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca70c26167a447ebaa59ab3504897b5" is_pre_election: true
I20260810 06:38:19.684238 13253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0489d81e52e4b1491d3ce597aa1a340" candidate_uuid: "afe31bcaaa2e4bf5a9251629035d2264" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d6785149bb9486b983950d2accae449" is_pre_election: true
I20260810 06:38:19.684455 12779 raft_consensus.cc:2454] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afe31bcaaa2e4bf5a9251629035d2264 in term 0.
I20260810 06:38:19.684468 13253 raft_consensus.cc:2454] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afe31bcaaa2e4bf5a9251629035d2264 in term 0.
I20260810 06:38:19.684868 12914 leader_election.cc:302] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ca70c26167a447ebaa59ab3504897b5, afe31bcaaa2e4bf5a9251629035d2264; no voters:
I20260810 06:38:19.685143 13420 raft_consensus.cc:2781] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:38:19.685259 13420 raft_consensus.cc:482] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:38:19.685374 13420 raft_consensus.cc:3037] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:19.687346 13420 raft_consensus.cc:504] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.687923 13420 leader_election.cc:288] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [CANDIDATE]: Term 1 election: Requested vote from peers 7ca70c26167a447ebaa59ab3504897b5 (127.31.59.193:40419), 1d6785149bb9486b983950d2accae449 (127.31.59.195:42567)
I20260810 06:38:19.688369 12779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0489d81e52e4b1491d3ce597aa1a340" candidate_uuid: "afe31bcaaa2e4bf5a9251629035d2264" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca70c26167a447ebaa59ab3504897b5"
I20260810 06:38:19.688578 12779 raft_consensus.cc:3037] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:19.688486 13253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0489d81e52e4b1491d3ce597aa1a340" candidate_uuid: "afe31bcaaa2e4bf5a9251629035d2264" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d6785149bb9486b983950d2accae449"
I20260810 06:38:19.688714 13253 raft_consensus.cc:3037] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:19.690763 13253 raft_consensus.cc:2454] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afe31bcaaa2e4bf5a9251629035d2264 in term 1.
I20260810 06:38:19.690773 12779 raft_consensus.cc:2454] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afe31bcaaa2e4bf5a9251629035d2264 in term 1.
I20260810 06:38:19.691172 12908 leader_election.cc:302] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d6785149bb9486b983950d2accae449, afe31bcaaa2e4bf5a9251629035d2264; no voters:
I20260810 06:38:19.691422 13420 raft_consensus.cc:2781] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:19.691803 13420 raft_consensus.cc:686] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [term 1 LEADER]: Becoming Leader. State: Replica: afe31bcaaa2e4bf5a9251629035d2264, State: Running, Role: LEADER
I20260810 06:38:19.692072 13420 consensus_queue.cc:237] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:19.693511 13473 tablet_replica.cc:439] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264: TxnStatusTablet state changed. Reason: New leader afe31bcaaa2e4bf5a9251629035d2264. Latest consensus state: current_term: 1 leader_uuid: "afe31bcaaa2e4bf5a9251629035d2264" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } }
I20260810 06:38:19.693818 13473 tablet_replica.cc:442] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:38:19.694392 13479 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 06:38:19.694605 13479 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 06:38:19.694931 12505 catalog_manager.cc:5495] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 reported cstate change: term changed from 0 to 1, leader changed from <none> to afe31bcaaa2e4bf5a9251629035d2264 (127.31.59.194). New cstate: current_term: 1 leader_uuid: "afe31bcaaa2e4bf5a9251629035d2264" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } health_report { overall_health: UNKNOWN } } }
I20260810 06:38:19.701860 13413 consensus_queue.cc:1035] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:19.707773 13413 consensus_queue.cc:1035] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:19.759421 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.752741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:35001 (local address 127.0.0.1:42208)
0810 06:38:19.753024 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:19.753049 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:19.753216 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:19.753704 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:19.753708 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:19.753717 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:38:19.753871 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.753878 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.755452 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.755458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.757002 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.757018 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.758038 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.758044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.758281 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.758386 (+ 105us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:38:19.758841 (+ 455us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:38:19.758861 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 06:38:19.758940 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 06:38:19.759111 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 06:38:19.759456 13491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.752708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42208 (local address 127.31.59.194:35001)
0810 06:38:19.753381 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.753387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.753407 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:38:19.753476 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.753481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.753588 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.753689 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:19.754003 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.755278 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.757149 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.757887 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.758010 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.758611 (+ 601us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:19.758742 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:38:19.758829 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.758978 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.759061 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":398,"threads_started":1}
I20260810 06:38:19.763887 12779 raft_consensus.cc:1261] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5 [term 1 FOLLOWER]: Refusing update from remote peer afe31bcaaa2e4bf5a9251629035d2264: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:19.764251 13253 raft_consensus.cc:1261] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449 [term 1 FOLLOWER]: Refusing update from remote peer afe31bcaaa2e4bf5a9251629035d2264: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:19.764645 13473 consensus_queue.cc:1035] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:19.765152 13420 consensus_queue.cc:1035] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:19.779119 13414 tablet_replica.cc:439] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449: TxnStatusTablet state changed. Reason: New leader afe31bcaaa2e4bf5a9251629035d2264. Latest consensus state: current_term: 1 leader_uuid: "afe31bcaaa2e4bf5a9251629035d2264" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } }
I20260810 06:38:19.779381 13414 tablet_replica.cc:442] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:38:19.780999 13483 mvcc.cc:205] Tried to move back new op lower bound from 7316864613426765824 to 7316864613140791296. Current Snapshot: MvccSnapshot[applied={T|T < 7316864613426765824}]
I20260810 06:38:19.782450 13486 mvcc.cc:205] Tried to move back new op lower bound from 7316864613426765824 to 7316864613140791296. Current Snapshot: MvccSnapshot[applied={T|T < 7316864613426765824}]
I20260810 06:38:19.782866 13484 tablet_replica.cc:439] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5: TxnStatusTablet state changed. Reason: New leader afe31bcaaa2e4bf5a9251629035d2264. Latest consensus state: current_term: 1 leader_uuid: "afe31bcaaa2e4bf5a9251629035d2264" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } }
I20260810 06:38:19.783183 13484 tablet_replica.cc:442] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:38:19.783286 13420 tablet_replica.cc:439] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "afe31bcaaa2e4bf5a9251629035d2264" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } }
I20260810 06:38:19.783582 13420 tablet_replica.cc:442] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:38:19.786760 13414 tablet_replica.cc:439] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "afe31bcaaa2e4bf5a9251629035d2264" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } }
I20260810 06:38:19.787022 13414 tablet_replica.cc:442] T a0489d81e52e4b1491d3ce597aa1a340 P 1d6785149bb9486b983950d2accae449: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:38:19.790038 13420 tablet_replica.cc:439] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "afe31bcaaa2e4bf5a9251629035d2264" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } }
I20260810 06:38:19.790343 13420 tablet_replica.cc:442] T a0489d81e52e4b1491d3ce597aa1a340 P afe31bcaaa2e4bf5a9251629035d2264: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:38:19.796058 13484 tablet_replica.cc:439] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "afe31bcaaa2e4bf5a9251629035d2264" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } } }
I20260810 06:38:19.796345 13484 tablet_replica.cc:442] T a0489d81e52e4b1491d3ce597aa1a340 P 7ca70c26167a447ebaa59ab3504897b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:38:19.804853 13506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.798142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43846 (local address 127.31.59.193:40419)
0810 06:38:19.798686 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.798692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.798895 (+ 203us) server_negotiation.cc:407] Connection header received
0810 06:38:19.799115 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.799119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.799201 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.799335 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:19.799726 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.800558 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.802509 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.802948 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.803046 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.803563 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:19.804253 (+ 690us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:38:19.804403 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.804500 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.804566 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":268,"threads_started":1}
I20260810 06:38:19.820601 13491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.814862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:45845 (local address 127.31.59.194:35001)
0810 06:38:19.815074 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.815082 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.815507 (+ 425us) server_negotiation.cc:407] Connection header received
0810 06:38:19.815716 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.815720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.815793 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.815859 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:19.816300 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.817028 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.819028 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.819439 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.819530 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.819989 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:19.820089 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:38:19.820154 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.820300 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.820368 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:38:19.820798 13510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.814775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:35001 (local address 127.31.59.193:45845)
0810 06:38:19.815403 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:19.815422 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:19.815575 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:19.815894 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:19.815898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:19.815911 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:38:19.816163 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.816175 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.817157 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.817163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.818900 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.818916 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.819587 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.819594 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.819767 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.819835 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:38:19.820180 (+ 345us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:38:19.820191 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 06:38:19.820265 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 06:38:19.820411 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":315,"threads_started":1}
I20260810 06:38:19.860512 13506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.853957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:39499 (local address 127.31.59.193:40419)
0810 06:38:19.854111 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.854115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.854782 (+ 667us) server_negotiation.cc:407] Connection header received
0810 06:38:19.854964 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.854968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.855030 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.855100 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:19.855596 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.856576 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.858654 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.859120 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.859242 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.859806 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:19.859931 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:38:19.860009 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.860154 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.860232 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 06:38:19.860632 13517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.854055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:40419 (local address 127.31.59.194:39499)
0810 06:38:19.854682 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:19.854698 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:19.854841 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:19.855146 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:19.855151 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:19.855162 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:38:19.855444 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.855453 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.856720 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.856726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.858512 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.858529 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.859330 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.859337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.859521 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.859604 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:38:19.860027 (+ 423us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:38:19.860039 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 06:38:19.860115 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 06:38:19.860251 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":335,"threads_started":1}
I20260810 06:38:19.907847 12505 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:19.908139 12505 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:19.911752 12505 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc: Sending DeleteTablet for 3 replicas of tablet e61dd059135d409693b2df07ed1fc6f1
I20260810 06:38:19.912604 12751 tablet_service.cc:1514] Processing DeleteTablet for tablet e61dd059135d409693b2df07ed1fc6f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:19 UTC) from {username='oryx', principal='oryx/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:43838
I20260810 06:38:19.912880 13216 tablet_service.cc:1514] Processing DeleteTablet for tablet e61dd059135d409693b2df07ed1fc6f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:19 UTC) from {username='oryx', principal='oryx/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:46830
I20260810 06:38:19.914885 12983 tablet_service.cc:1514] Processing DeleteTablet for tablet e61dd059135d409693b2df07ed1fc6f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:19 UTC) from {username='oryx', principal='oryx/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:42190
I20260810 06:38:19.922766 13524 tablet_replica.cc:330] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5: stopping tablet replica
I20260810 06:38:19.923138 13524 raft_consensus.cc:2229] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:38:19.923854 13524 raft_consensus.cc:2258] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:19.925120 13524 ts_tablet_manager.cc:1905] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:19.933346 13524 ts_tablet_manager.cc:1918] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:38:19.933590 13524 log.cc:1196] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/e61dd059135d409693b2df07ed1fc6f1
I20260810 06:38:19.934367 13524 ts_tablet_manager.cc:1939] T e61dd059135d409693b2df07ed1fc6f1 P 7ca70c26167a447ebaa59ab3504897b5: Deleting consensus metadata
I20260810 06:38:19.935660 12486 catalog_manager.cc:4841] TS 7ca70c26167a447ebaa59ab3504897b5 (127.31.59.193:40419): tablet e61dd059135d409693b2df07ed1fc6f1 (table test-table [id=c17c27f9b86346709ab067378e188008]) successfully deleted
I20260810 06:38:19.941629 13527 tablet_replica.cc:330] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264: stopping tablet replica
I20260810 06:38:19.941937 13527 raft_consensus.cc:2229] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:19.942528 13525 tablet_replica.cc:330] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449: stopping tablet replica
I20260810 06:38:19.942857 13527 raft_consensus.cc:2258] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:19.943127 13525 raft_consensus.cc:2229] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:19.944402 13527 ts_tablet_manager.cc:1905] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:19.944737 13525 raft_consensus.cc:2258] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:19.945906 13525 ts_tablet_manager.cc:1905] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:19.954010 13527 ts_tablet_manager.cc:1918] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:38:19.961668 13525 ts_tablet_manager.cc:1918] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:38:19.961907 13525 log.cc:1196] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/e61dd059135d409693b2df07ed1fc6f1
I20260810 06:38:19.962103 13527 log.cc:1196] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/e61dd059135d409693b2df07ed1fc6f1
I20260810 06:38:19.962654 13525 ts_tablet_manager.cc:1939] T e61dd059135d409693b2df07ed1fc6f1 P 1d6785149bb9486b983950d2accae449: Deleting consensus metadata
I20260810 06:38:19.963738 13527 ts_tablet_manager.cc:1939] T e61dd059135d409693b2df07ed1fc6f1 P afe31bcaaa2e4bf5a9251629035d2264: Deleting consensus metadata
I20260810 06:38:19.965004 12481 catalog_manager.cc:4841] TS 1d6785149bb9486b983950d2accae449 (127.31.59.195:42567): tablet e61dd059135d409693b2df07ed1fc6f1 (table test-table [id=c17c27f9b86346709ab067378e188008]) successfully deleted
I20260810 06:38:19.966485 12486 catalog_manager.cc:4841] TS afe31bcaaa2e4bf5a9251629035d2264 (127.31.59.194:35001): tablet e61dd059135d409693b2df07ed1fc6f1 (table test-table [id=c17c27f9b86346709ab067378e188008]) successfully deleted
Aug 10 06:38:19 dist-test-slave-4tdj krb5kdc[12435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:38:20 dist-test-slave-4tdj krb5kdc[12435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:20 dist-test-slave-4tdj krb5kdc[12435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:38:20.021611 13539 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:19.998343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38036 (local address 127.31.59.254:40097)
0810 06:38:20.011385 (+ 13042us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.011391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.011415 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:38:20.011484 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.011488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.011558 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.011745 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:20.013311 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.014111 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.015448 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.015897 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.016037 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.021317 (+ 5280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38036: BlockingRecv error: recv got EOF from 127.0.0.1:38036 (error 108)
Metrics: {"server-negotiator.queue_time_us":12292,"thread_start_us":12134,"threads_started":1}
Aug 10 06:38:20 dist-test-slave-4tdj krb5kdc[12435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM
I20260810 06:38:20.045827 13539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.031599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38040 (local address 127.31.59.254:40097)
0810 06:38:20.031797 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.031804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.032091 (+ 287us) server_negotiation.cc:407] Connection header received
0810 06:38:20.032259 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.032265 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.032339 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.032421 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:20.033738 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.034543 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.035803 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.036243 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.036355 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.040425 (+ 4070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:20.040431 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:20.040434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:20.040471 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:20.043190 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:20.043849 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:20.043855 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:20.043859 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:20.044000 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:20.044366 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:20.044370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:20.044372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:20.044652 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:20.044890 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:20.045190 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 06:38:20.045425 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:38:20.049629 12505 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38040:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:38:20.050251 12505 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:38:20.057461 13216 tablet_service.cc:1467] Processing CreateTablet for tablet 906385beb0284896b6cd84d9fe9939bc (DEFAULT_TABLE table=test-table [id=dd503e58ddce4fe9a78629c1977a90ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:20.057967 13216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 906385beb0284896b6cd84d9fe9939bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:20.058200 12751 tablet_service.cc:1467] Processing CreateTablet for tablet 906385beb0284896b6cd84d9fe9939bc (DEFAULT_TABLE table=test-table [id=dd503e58ddce4fe9a78629c1977a90ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:20.058655 12751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 906385beb0284896b6cd84d9fe9939bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:20.062740 12983 tablet_service.cc:1467] Processing CreateTablet for tablet 906385beb0284896b6cd84d9fe9939bc (DEFAULT_TABLE table=test-table [id=dd503e58ddce4fe9a78629c1977a90ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:20.064427 12983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 906385beb0284896b6cd84d9fe9939bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:20.069515 13551 tablet_bootstrap.cc:492] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5: Bootstrap starting.
I20260810 06:38:20.071712 13551 tablet_bootstrap.cc:654] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:20.072369 13553 tablet_bootstrap.cc:492] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449: Bootstrap starting.
I20260810 06:38:20.074141 13553 tablet_bootstrap.cc:654] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:20.080287 13551 tablet_bootstrap.cc:492] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5: No bootstrap required, opened a new log
I20260810 06:38:20.080456 13551 ts_tablet_manager.cc:1397] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.007s
I20260810 06:38:20.081312 13551 raft_consensus.cc:348] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:20.081569 13551 raft_consensus.cc:374] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:20.081665 13551 raft_consensus.cc:729] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca70c26167a447ebaa59ab3504897b5, State: Initialized, Role: FOLLOWER
I20260810 06:38:20.081951 13551 consensus_queue.cc:260] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:20.082605 13551 ts_tablet_manager.cc:1428] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 06:38:20.085001 13553 tablet_bootstrap.cc:492] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449: No bootstrap required, opened a new log
I20260810 06:38:20.085238 13553 ts_tablet_manager.cc:1397] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260810 06:38:20.086143 13553 raft_consensus.cc:348] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:20.086506 13553 raft_consensus.cc:374] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:20.086663 13553 raft_consensus.cc:729] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d6785149bb9486b983950d2accae449, State: Initialized, Role: FOLLOWER
I20260810 06:38:20.086970 13553 consensus_queue.cc:260] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:20.087672 13553 ts_tablet_manager.cc:1428] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:38:20.091600 13556 tablet_bootstrap.cc:492] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264: Bootstrap starting.
I20260810 06:38:20.093618 13556 tablet_bootstrap.cc:654] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:20.095970 13556 tablet_bootstrap.cc:492] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264: No bootstrap required, opened a new log
I20260810 06:38:20.096122 13556 ts_tablet_manager.cc:1397] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260810 06:38:20.096822 13556 raft_consensus.cc:348] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:20.097020 13556 raft_consensus.cc:374] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:20.097112 13556 raft_consensus.cc:729] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afe31bcaaa2e4bf5a9251629035d2264, State: Initialized, Role: FOLLOWER
I20260810 06:38:20.097363 13556 consensus_queue.cc:260] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:20.098094 13556 ts_tablet_manager.cc:1428] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 06:38:20.319175 13521 raft_consensus.cc:482] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:20.319362 13521 raft_consensus.cc:504] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:20.320004 13521 leader_election.cc:288] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ca70c26167a447ebaa59ab3504897b5 (127.31.59.193:40419), 1d6785149bb9486b983950d2accae449 (127.31.59.195:42567)
I20260810 06:38:20.320596 12778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906385beb0284896b6cd84d9fe9939bc" candidate_uuid: "afe31bcaaa2e4bf5a9251629035d2264" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca70c26167a447ebaa59ab3504897b5" is_pre_election: true
I20260810 06:38:20.320680 13256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906385beb0284896b6cd84d9fe9939bc" candidate_uuid: "afe31bcaaa2e4bf5a9251629035d2264" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d6785149bb9486b983950d2accae449" is_pre_election: true
I20260810 06:38:20.320886 12778 raft_consensus.cc:2454] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afe31bcaaa2e4bf5a9251629035d2264 in term 0.
I20260810 06:38:20.320991 13256 raft_consensus.cc:2454] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afe31bcaaa2e4bf5a9251629035d2264 in term 0.
I20260810 06:38:20.321488 12908 leader_election.cc:302] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1d6785149bb9486b983950d2accae449, 7ca70c26167a447ebaa59ab3504897b5, afe31bcaaa2e4bf5a9251629035d2264; no voters:
I20260810 06:38:20.321765 13521 raft_consensus.cc:2781] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:38:20.321883 13521 raft_consensus.cc:482] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:38:20.321992 13521 raft_consensus.cc:3037] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:20.324507 13521 raft_consensus.cc:504] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:20.325026 13521 leader_election.cc:288] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [CANDIDATE]: Term 1 election: Requested vote from peers 7ca70c26167a447ebaa59ab3504897b5 (127.31.59.193:40419), 1d6785149bb9486b983950d2accae449 (127.31.59.195:42567)
I20260810 06:38:20.325537 13256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906385beb0284896b6cd84d9fe9939bc" candidate_uuid: "afe31bcaaa2e4bf5a9251629035d2264" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d6785149bb9486b983950d2accae449"
I20260810 06:38:20.325621 12778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906385beb0284896b6cd84d9fe9939bc" candidate_uuid: "afe31bcaaa2e4bf5a9251629035d2264" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca70c26167a447ebaa59ab3504897b5"
I20260810 06:38:20.325776 13256 raft_consensus.cc:3037] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:20.325845 12778 raft_consensus.cc:3037] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:20.328819 13256 raft_consensus.cc:2454] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afe31bcaaa2e4bf5a9251629035d2264 in term 1.
I20260810 06:38:20.328948 12778 raft_consensus.cc:2454] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afe31bcaaa2e4bf5a9251629035d2264 in term 1.
I20260810 06:38:20.329386 12908 leader_election.cc:302] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d6785149bb9486b983950d2accae449, afe31bcaaa2e4bf5a9251629035d2264; no voters:
I20260810 06:38:20.329686 13521 raft_consensus.cc:2781] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:20.329818 13521 raft_consensus.cc:686] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [term 1 LEADER]: Becoming Leader. State: Replica: afe31bcaaa2e4bf5a9251629035d2264, State: Running, Role: LEADER
I20260810 06:38:20.330107 13521 consensus_queue.cc:237] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:20.332808 12503 catalog_manager.cc:5495] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 reported cstate change: term changed from 0 to 1, leader changed from <none> to afe31bcaaa2e4bf5a9251629035d2264 (127.31.59.194). New cstate: current_term: 1 leader_uuid: "afe31bcaaa2e4bf5a9251629035d2264" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } health_report { overall_health: UNKNOWN } } }
I20260810 06:38:20.379179 13566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.374360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42216 (local address 127.31.59.194:35001)
0810 06:38:20.374861 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.374867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.374883 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:38:20.374944 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.374948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.375009 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.375090 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:20.375457 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.376197 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.377489 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.377903 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.378012 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.378467 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:20.378594 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:38:20.378670 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:20.378812 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 06:38:20.378892 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":258,"threads_started":1}
I20260810 06:38:20.381793 13256 raft_consensus.cc:1261] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449 [term 1 FOLLOWER]: Refusing update from remote peer afe31bcaaa2e4bf5a9251629035d2264: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:20.381796 12778 raft_consensus.cc:1261] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5 [term 1 FOLLOWER]: Refusing update from remote peer afe31bcaaa2e4bf5a9251629035d2264: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:20.382400 13473 consensus_queue.cc:1035] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:20.382676 13521 consensus_queue.cc:1035] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:20.407024 12505 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:20.407294 12505 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:20.410161 12505 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc: Sending DeleteTablet for 3 replicas of tablet 906385beb0284896b6cd84d9fe9939bc
I20260810 06:38:20.410928 12751 tablet_service.cc:1514] Processing DeleteTablet for tablet 906385beb0284896b6cd84d9fe9939bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:20 UTC) from {username='oryx', principal='oryx/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:43838
I20260810 06:38:20.410987 12983 tablet_service.cc:1514] Processing DeleteTablet for tablet 906385beb0284896b6cd84d9fe9939bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:20 UTC) from {username='oryx', principal='oryx/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:42190
I20260810 06:38:20.411154 13216 tablet_service.cc:1514] Processing DeleteTablet for tablet 906385beb0284896b6cd84d9fe9939bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:20 UTC) from {username='oryx', principal='oryx/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:46830
I20260810 06:38:20.411222 13524 tablet_replica.cc:330] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5: stopping tablet replica
I20260810 06:38:20.411376 13527 tablet_replica.cc:330] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264: stopping tablet replica
I20260810 06:38:20.411423 13524 raft_consensus.cc:2229] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:20.411602 13527 raft_consensus.cc:2229] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:38:20.411629 13524 raft_consensus.cc:2258] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:20.411880 13525 tablet_replica.cc:330] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449: stopping tablet replica
I20260810 06:38:20.411901 13527 raft_consensus.cc:2258] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:20.412096 13525 raft_consensus.cc:2229] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:20.412310 13525 raft_consensus.cc:2258] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:20.412451 12505 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38040:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 06:38:20.413146 13527 ts_tablet_manager.cc:1905] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 06:38:20.413168 12505 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:38:20.413319 13524 ts_tablet_manager.cc:1905] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:20.417666 13527 ts_tablet_manager.cc:1918] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:20.417775 13527 log.cc:1196] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/906385beb0284896b6cd84d9fe9939bc
I20260810 06:38:20.418059 13527 ts_tablet_manager.cc:1939] T 906385beb0284896b6cd84d9fe9939bc P afe31bcaaa2e4bf5a9251629035d2264: Deleting consensus metadata
I20260810 06:38:20.419438 12486 catalog_manager.cc:4841] TS afe31bcaaa2e4bf5a9251629035d2264 (127.31.59.194:35001): tablet 906385beb0284896b6cd84d9fe9939bc (table test-table [id=dd503e58ddce4fe9a78629c1977a90ee]) successfully deleted
I20260810 06:38:20.419845 13524 ts_tablet_manager.cc:1918] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:20.419996 13524 log.cc:1196] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/906385beb0284896b6cd84d9fe9939bc
I20260810 06:38:20.420388 13524 ts_tablet_manager.cc:1939] T 906385beb0284896b6cd84d9fe9939bc P 7ca70c26167a447ebaa59ab3504897b5: Deleting consensus metadata
I20260810 06:38:20.422088 13525 ts_tablet_manager.cc:1905] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:20.422197 12486 catalog_manager.cc:4841] TS 7ca70c26167a447ebaa59ab3504897b5 (127.31.59.193:40419): tablet 906385beb0284896b6cd84d9fe9939bc (table test-table [id=dd503e58ddce4fe9a78629c1977a90ee]) successfully deleted
I20260810 06:38:20.425350 13216 tablet_service.cc:1467] Processing CreateTablet for tablet 757f46cac6664c6bbc8c177b8a61feca (DEFAULT_TABLE table=test-table [id=98a4979e0c574095bd1920800c07c8bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:20.425347 12751 tablet_service.cc:1467] Processing CreateTablet for tablet 757f46cac6664c6bbc8c177b8a61feca (DEFAULT_TABLE table=test-table [id=98a4979e0c574095bd1920800c07c8bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:20.425786 13216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 757f46cac6664c6bbc8c177b8a61feca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:20.425784 12751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 757f46cac6664c6bbc8c177b8a61feca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:20.428547 13525 ts_tablet_manager.cc:1918] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:20.428684 13525 log.cc:1196] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/906385beb0284896b6cd84d9fe9939bc
I20260810 06:38:20.429071 13525 ts_tablet_manager.cc:1939] T 906385beb0284896b6cd84d9fe9939bc P 1d6785149bb9486b983950d2accae449: Deleting consensus metadata
I20260810 06:38:20.430174 12481 catalog_manager.cc:4841] TS 1d6785149bb9486b983950d2accae449 (127.31.59.195:42567): tablet 906385beb0284896b6cd84d9fe9939bc (table test-table [id=dd503e58ddce4fe9a78629c1977a90ee]) successfully deleted
I20260810 06:38:20.430984 12983 tablet_service.cc:1467] Processing CreateTablet for tablet 757f46cac6664c6bbc8c177b8a61feca (DEFAULT_TABLE table=test-table [id=98a4979e0c574095bd1920800c07c8bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:20.431386 12983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 757f46cac6664c6bbc8c177b8a61feca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:20.432379 13551 tablet_bootstrap.cc:492] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5: Bootstrap starting.
I20260810 06:38:20.432726 13553 tablet_bootstrap.cc:492] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449: Bootstrap starting.
I20260810 06:38:20.433949 13551 tablet_bootstrap.cc:654] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:20.434140 13553 tablet_bootstrap.cc:654] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:20.436005 13553 tablet_bootstrap.cc:492] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449: No bootstrap required, opened a new log
I20260810 06:38:20.436131 13553 ts_tablet_manager.cc:1397] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.002s
I20260810 06:38:20.436673 13553 raft_consensus.cc:348] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:20.436849 13553 raft_consensus.cc:374] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:20.436898 13553 raft_consensus.cc:729] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d6785149bb9486b983950d2accae449, State: Initialized, Role: FOLLOWER
I20260810 06:38:20.437124 13553 consensus_queue.cc:260] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:20.437530 13556 tablet_bootstrap.cc:492] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264: Bootstrap starting.
I20260810 06:38:20.437593 13553 ts_tablet_manager.cc:1428] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 06:38:20.439124 13556 tablet_bootstrap.cc:654] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:20.439519 13551 tablet_bootstrap.cc:492] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5: No bootstrap required, opened a new log
I20260810 06:38:20.440451 13551 ts_tablet_manager.cc:1397] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260810 06:38:20.441169 13551 raft_consensus.cc:348] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:20.441362 13551 raft_consensus.cc:374] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:20.441433 13551 raft_consensus.cc:729] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca70c26167a447ebaa59ab3504897b5, State: Initialized, Role: FOLLOWER
I20260810 06:38:20.441679 13551 consensus_queue.cc:260] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:20.442204 13551 ts_tablet_manager.cc:1428] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 06:38:20.442283 13556 tablet_bootstrap.cc:492] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264: No bootstrap required, opened a new log
I20260810 06:38:20.442464 13556 ts_tablet_manager.cc:1397] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260810 06:38:20.443136 13556 raft_consensus.cc:348] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:20.443361 13556 raft_consensus.cc:374] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:20.443439 13556 raft_consensus.cc:729] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afe31bcaaa2e4bf5a9251629035d2264, State: Initialized, Role: FOLLOWER
I20260810 06:38:20.443681 13556 consensus_queue.cc:260] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:20.444672 13556 ts_tablet_manager.cc:1428] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:38:20.636818 13414 raft_consensus.cc:482] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:20.637013 13414 raft_consensus.cc:504] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:20.637588 13414 leader_election.cc:288] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ca70c26167a447ebaa59ab3504897b5 (127.31.59.193:40419), afe31bcaaa2e4bf5a9251629035d2264 (127.31.59.194:35001)
I20260810 06:38:20.638247 12778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "757f46cac6664c6bbc8c177b8a61feca" candidate_uuid: "1d6785149bb9486b983950d2accae449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca70c26167a447ebaa59ab3504897b5" is_pre_election: true
I20260810 06:38:20.638326 13024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "757f46cac6664c6bbc8c177b8a61feca" candidate_uuid: "1d6785149bb9486b983950d2accae449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afe31bcaaa2e4bf5a9251629035d2264" is_pre_election: true
I20260810 06:38:20.638549 12778 raft_consensus.cc:2454] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d6785149bb9486b983950d2accae449 in term 0.
I20260810 06:38:20.638563 13024 raft_consensus.cc:2454] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d6785149bb9486b983950d2accae449 in term 0.
I20260810 06:38:20.638962 13142 leader_election.cc:302] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d6785149bb9486b983950d2accae449, 7ca70c26167a447ebaa59ab3504897b5; no voters:
I20260810 06:38:20.639248 13414 raft_consensus.cc:2781] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:38:20.639359 13414 raft_consensus.cc:482] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:38:20.639446 13414 raft_consensus.cc:3037] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:20.641567 13414 raft_consensus.cc:504] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:20.642091 13414 leader_election.cc:288] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [CANDIDATE]: Term 1 election: Requested vote from peers 7ca70c26167a447ebaa59ab3504897b5 (127.31.59.193:40419), afe31bcaaa2e4bf5a9251629035d2264 (127.31.59.194:35001)
I20260810 06:38:20.642588 12778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "757f46cac6664c6bbc8c177b8a61feca" candidate_uuid: "1d6785149bb9486b983950d2accae449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca70c26167a447ebaa59ab3504897b5"
I20260810 06:38:20.642634 13024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "757f46cac6664c6bbc8c177b8a61feca" candidate_uuid: "1d6785149bb9486b983950d2accae449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afe31bcaaa2e4bf5a9251629035d2264"
I20260810 06:38:20.642771 12778 raft_consensus.cc:3037] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:20.642798 13024 raft_consensus.cc:3037] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:20.644695 12778 raft_consensus.cc:2454] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d6785149bb9486b983950d2accae449 in term 1.
I20260810 06:38:20.644752 13024 raft_consensus.cc:2454] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d6785149bb9486b983950d2accae449 in term 1.
I20260810 06:38:20.645087 13142 leader_election.cc:302] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d6785149bb9486b983950d2accae449, 7ca70c26167a447ebaa59ab3504897b5; no voters:
I20260810 06:38:20.645386 13414 raft_consensus.cc:2781] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:20.645794 13414 raft_consensus.cc:686] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [term 1 LEADER]: Becoming Leader. State: Replica: 1d6785149bb9486b983950d2accae449, State: Running, Role: LEADER
I20260810 06:38:20.646193 13414 consensus_queue.cc:237] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } }
I20260810 06:38:20.649376 12503 catalog_manager.cc:5495] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d6785149bb9486b983950d2accae449 (127.31.59.195). New cstate: current_term: 1 leader_uuid: "1d6785149bb9486b983950d2accae449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d6785149bb9486b983950d2accae449" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42567 } health_report { overall_health: HEALTHY } } }
I20260810 06:38:20.686820 13582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.681042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46840 (local address 127.31.59.195:42567)
0810 06:38:20.681584 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.681589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.681606 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:38:20.681681 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.681684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.681757 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.681899 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:20.682350 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.683130 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.684430 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.684899 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.685107 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.685546 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:20.686140 (+ 594us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:38:20.686337 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:20.686410 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 06:38:20.686507 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":274,"threads_started":1}
I20260810 06:38:20.701020 13566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.696320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:36113 (local address 127.31.59.194:35001)
0810 06:38:20.696542 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.696548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.696883 (+ 335us) server_negotiation.cc:407] Connection header received
0810 06:38:20.696998 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.697002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.697068 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.697139 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:20.697589 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.698320 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.699606 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.699995 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.700084 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.700411 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:20.700542 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:38:20.700605 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:20.700766 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 06:38:20.700827 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:38:20.701154 13585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.696186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:35001 (local address 127.31.59.195:36113)
0810 06:38:20.696775 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:20.696791 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:20.696900 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:20.697153 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:20.697156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:20.697164 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:38:20.697393 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.697400 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.698440 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.698444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.699471 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.699493 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.700124 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.700129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.700236 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.700285 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:38:20.700653 (+ 368us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:38:20.700666 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 06:38:20.700745 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 06:38:20.700846 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":320,"threads_started":1}
I20260810 06:38:20.709367 13024 raft_consensus.cc:1261] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264 [term 1 FOLLOWER]: Refusing update from remote peer 1d6785149bb9486b983950d2accae449: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:20.709432 12778 raft_consensus.cc:1261] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5 [term 1 FOLLOWER]: Refusing update from remote peer 1d6785149bb9486b983950d2accae449: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:20.709952 13580 consensus_queue.cc:1035] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ca70c26167a447ebaa59ab3504897b5" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:20.710280 13414 consensus_queue.cc:1035] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afe31bcaaa2e4bf5a9251629035d2264" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:20.737310 13582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.732886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:55129 (local address 127.31.59.195:42567)
0810 06:38:20.733041 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.733046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.733180 (+ 134us) server_negotiation.cc:407] Connection header received
0810 06:38:20.733323 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.733326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.733392 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.733456 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:20.733792 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.734569 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.735862 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.736265 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.736384 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.736719 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:20.736817 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:38:20.736881 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:20.737061 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 06:38:20.737124 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:38:20.737390 13590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.732589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:42567 (local address 127.31.59.194:55129)
0810 06:38:20.733090 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:20.733111 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:20.733238 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:20.733462 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:20.733465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:20.733473 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:38:20.733687 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.733695 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.734698 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.734703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.735728 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.735741 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.736405 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.736410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.736550 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.736589 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:38:20.736928 (+ 339us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:38:20.736956 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 06:38:20.737038 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 06:38:20.737141 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":230,"threads_started":1}
I20260810 06:38:20.770640 12507 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:20.770897 12507 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:20.774307 12507 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45a743dbe05b4058a24dfe33b83883cc: Sending DeleteTablet for 3 replicas of tablet 757f46cac6664c6bbc8c177b8a61feca
I20260810 06:38:20.775221 13216 tablet_service.cc:1514] Processing DeleteTablet for tablet 757f46cac6664c6bbc8c177b8a61feca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:20 UTC) from {username='oryx', principal='oryx/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:46830
I20260810 06:38:20.775293 12751 tablet_service.cc:1514] Processing DeleteTablet for tablet 757f46cac6664c6bbc8c177b8a61feca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:20 UTC) from {username='oryx', principal='oryx/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:43838
I20260810 06:38:20.775574 13525 tablet_replica.cc:330] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449: stopping tablet replica
I20260810 06:38:20.775532 12983 tablet_service.cc:1514] Processing DeleteTablet for tablet 757f46cac6664c6bbc8c177b8a61feca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:20 UTC) from {username='oryx', principal='oryx/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:42190
I20260810 06:38:20.775595 13524 tablet_replica.cc:330] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5: stopping tablet replica
I20260810 06:38:20.775758 13525 raft_consensus.cc:2229] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:38:20.775764 13524 raft_consensus.cc:2229] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:20.775983 13524 raft_consensus.cc:2258] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:20.776198 13527 tablet_replica.cc:330] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264: stopping tablet replica
I20260810 06:38:20.776248 13525 raft_consensus.cc:2258] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:20.776363 13527 raft_consensus.cc:2229] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:20.776564 13527 raft_consensus.cc:2258] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:20.777202 13524 ts_tablet_manager.cc:1905] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:20.778648 13525 ts_tablet_manager.cc:1905] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:20.782663 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 12632
I20260810 06:38:20.783676 13524 ts_tablet_manager.cc:1918] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:38:20.783788 13524 log.cc:1196] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/757f46cac6664c6bbc8c177b8a61feca
I20260810 06:38:20.784183 13524 ts_tablet_manager.cc:1939] T 757f46cac6664c6bbc8c177b8a61feca P 7ca70c26167a447ebaa59ab3504897b5: Deleting consensus metadata
I20260810 06:38:20.785775 12486 catalog_manager.cc:4841] TS 7ca70c26167a447ebaa59ab3504897b5 (127.31.59.193:40419): tablet 757f46cac6664c6bbc8c177b8a61feca (table test-table [id=98a4979e0c574095bd1920800c07c8bb]) successfully deleted
I20260810 06:38:20.791688 13525 ts_tablet_manager.cc:1918] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:38:20.791838 13525 log.cc:1196] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/757f46cac6664c6bbc8c177b8a61feca
I20260810 06:38:20.792287 13525 ts_tablet_manager.cc:1939] T 757f46cac6664c6bbc8c177b8a61feca P 1d6785149bb9486b983950d2accae449: Deleting consensus metadata
I20260810 06:38:20.793339 12481 catalog_manager.cc:4841] TS 1d6785149bb9486b983950d2accae449 (127.31.59.195:42567): tablet 757f46cac6664c6bbc8c177b8a61feca (table test-table [id=98a4979e0c574095bd1920800c07c8bb]) successfully deleted
I20260810 06:38:20.794436 13527 ts_tablet_manager.cc:1905] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:20.799585 13527 ts_tablet_manager.cc:1918] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:38:20.799713 13527 log.cc:1196] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/757f46cac6664c6bbc8c177b8a61feca
I20260810 06:38:20.800040 13527 ts_tablet_manager.cc:1939] T 757f46cac6664c6bbc8c177b8a61feca P afe31bcaaa2e4bf5a9251629035d2264: Deleting consensus metadata
I20260810 06:38:20.801170 12486 catalog_manager.cc:4841] TS afe31bcaaa2e4bf5a9251629035d2264 (127.31.59.194:35001): tablet 757f46cac6664c6bbc8c177b8a61feca (table test-table [id=98a4979e0c574095bd1920800c07c8bb]) successfully deleted
Aug 10 06:38:20 dist-test-slave-4tdj krb5kdc[12435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.193@KRBTEST.COM
I20260810 06:38:20.811462 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.785502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43850 (local address 127.31.59.193:40419)
0810 06:38:20.795331 (+ 9829us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.795337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.795357 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:38:20.795416 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.795422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.795516 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.795628 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:20.797240 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.798342 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.800183 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.800820 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.800963 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.805758 (+ 4795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:20.805773 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:20.805790 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:20.805826 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:20.808500 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:20.809277 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:20.809284 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:20.809287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:20.809421 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:20.809859 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:20.809863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:20.809865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:20.810239 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:20.810568 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:20.810885 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 06:38:20.811117 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9700,"thread_start_us":9577,"threads_started":1}
I20260810 06:38:20.812167 12872 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:20.941676 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 12632
I20260810 06:38:20.978696 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 12884
Aug 10 06:38:20 dist-test-slave-4tdj krb5kdc[12435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.194@KRBTEST.COM
I20260810 06:38:20.998358 13566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.979563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42230 (local address 127.31.59.194:35001)
0810 06:38:20.979790 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.979797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.979818 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:38:20.979934 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.979940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.980041 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.980140 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:20.981427 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.982540 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.983863 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.984512 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.984642 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.991782 (+ 7140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:20.991799 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:20.991818 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:20.991862 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:20.995461 (+ 3599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:20.996156 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:20.996167 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:20.996170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:20.996336 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:20.996662 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:20.996667 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:20.996671 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:20.997165 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:20.997474 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:20.997725 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 06:38:20.997951 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 06:38:20.999059 13092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:21.145784 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 12884
I20260810 06:38:21.173934 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 13106
Aug 10 06:38:21 dist-test-slave-4tdj krb5kdc[12435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.59.195@KRBTEST.COM
I20260810 06:38:21.188912 13582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:21.174742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46852 (local address 127.31.59.195:42567)
0810 06:38:21.174950 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:21.174956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:21.174986 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:38:21.175052 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:21.175057 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:21.175141 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:21.175246 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:21.176545 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:21.177304 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:21.178595 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:21.179031 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:21.179143 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:21.183310 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:21.183322 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:21.183337 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:21.183370 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:21.186271 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:21.186917 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:21.186926 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:21.186930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:21.187075 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:21.187433 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:21.187439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:21.187442 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:21.187780 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:21.188058 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:21.188372 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 06:38:21.188597 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:38:21.189592 13327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:21.312923 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 13106
I20260810 06:38:21.346383 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 12453
I20260810 06:38:21.347705 12572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:21.495261 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 12453
2026-08-10T06:38:21Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7762 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:38:21 dist-test-slave-4tdj krb5kdc[13612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:38:21 dist-test-slave-4tdj krb5kdc[13612](info): set up 2 sockets
Aug 10 06:38:21 dist-test-slave-4tdj krb5kdc[13612](info): commencing operation
krb5kdc: starting...
W20260810 06:38:22.712374 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:38:22 dist-test-slave-4tdj krb5kdc[13612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:38:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:38:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:38:22.935949 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:44897
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:33153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:44897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:23.257308 13642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:23.257588 13642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:23.257726 13642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:23.264628 13642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:23.264731 13642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:23.264809 13642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:23.264886 13642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:23.264951 13642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:23.274240 13642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:33153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:44897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:44897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:23.275399 13642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:23.276628 13642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:23.287624 13654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:23.287664 13651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:23.287899 13650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:23.288591 13642 server_base.cc:1034] running on GCE node
I20260810 06:38:23.289279 13642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:23.290535 13642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:23.291731 13642 hybrid_clock.cc:648] HybridClock initialized: now 1786343903291679 us; error 41 us; skew 500 ppm
I20260810 06:38:23.292050 13642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T06:38:23Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1862 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:38:23 dist-test-slave-4tdj krb5kdc[13666](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:38:23 dist-test-slave-4tdj krb5kdc[13666](info): set up 2 sockets
Aug 10 06:38:23 dist-test-slave-4tdj krb5kdc[13666](info): commencing operation
krb5kdc: starting...
W20260810 06:38:24.616256 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:38:24 dist-test-slave-4tdj krb5kdc[13666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:38:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:38:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:38:24.833201 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40871
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:40871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:25.153314 13684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:25.153599 13684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:25.153689 13684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:25.161142 13684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:25.161244 13684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:25.161289 13684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:25.161345 13684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:25.161387 13684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:25.171065 13684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:40871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:25.172251 13684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:25.173477 13684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:25.184659 13695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:25.184823 13692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:25.184825 13698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:25.185999 13684 server_base.cc:1034] running on GCE node
I20260810 06:38:25.186834 13684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:25.188148 13684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:25.189337 13684 hybrid_clock.cc:648] HybridClock initialized: now 1786343905189290 us; error 35 us; skew 500 ppm
I20260810 06:38:25.189661 13684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:25 dist-test-slave-4tdj krb5kdc[13666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343905, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:25.192572 13684 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:38:25.193982 13684 webserver.cc:460] Webserver started at http://127.31.59.254:39337/ using document root <none> and password file <none>
I20260810 06:38:25.194690 13684 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:25.194805 13684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:25.195108 13684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:25.197899 13684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "42eb8185bf954e68aba245a3a0fbe47f"
format_stamp: "Formatted at 2026-08-10 06:38:25 on dist-test-slave-4tdj"
server_key: "d61c29fe021d6e060029f6533391deec"
server_key_iv: "580b61ecc7af755ece5ecfa08d111bf9"
server_key_version: "encryptionkey@0"
I20260810 06:38:25.198660 13684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "42eb8185bf954e68aba245a3a0fbe47f"
format_stamp: "Formatted at 2026-08-10 06:38:25 on dist-test-slave-4tdj"
server_key: "d61c29fe021d6e060029f6533391deec"
server_key_iv: "580b61ecc7af755ece5ecfa08d111bf9"
server_key_version: "encryptionkey@0"
I20260810 06:38:25.203481 13684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 06:38:25.206903 13707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:25.208228 13684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:38:25.208397 13684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "42eb8185bf954e68aba245a3a0fbe47f"
format_stamp: "Formatted at 2026-08-10 06:38:25 on dist-test-slave-4tdj"
server_key: "d61c29fe021d6e060029f6533391deec"
server_key_iv: "580b61ecc7af755ece5ecfa08d111bf9"
server_key_version: "encryptionkey@0"
I20260810 06:38:25.208570 13684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:25.224519 13684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:25.228375 13684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:25.228638 13684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:25.245777 13684 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:40871
I20260810 06:38:25.245800 13796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:40871 every 8 connection(s)
I20260810 06:38:25.247123 13684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:38:25.250711 13797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:25.251662 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 13684
I20260810 06:38:25.251906 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:38:25.252784 31983 external_mini_cluster.cc:1428] Setting key fc3603d42837442c2a03dc7919bbf4c6
I20260810 06:38:25.263233 13797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f: Bootstrap starting.
Aug 10 06:38:25 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:38:25.266438 13797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:25.267566 13797 log.cc:824] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:25.270474 13797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f: No bootstrap required, opened a new log
I20260810 06:38:25.273826 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:25.254858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32956 (local address 127.31.59.254:40871)
0810 06:38:25.255627 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:25.255638 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:25.255677 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:38:25.257302 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:25.257324 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:25.257697 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:25.258076 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:25.259515 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:25.260782 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:25.262009 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:25.262688 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:25.262852 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:25.266966 (+ 4114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:25.266974 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:25.266987 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:25.267020 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:25.269966 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:25.270800 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:25.270806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:25.270812 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:25.270984 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:25.271389 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:25.271392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:25.271394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:25.271870 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:25.272101 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:25.272533 (+ 432us) server_negotiation.cc:299] Negotiation successful
0810 06:38:25.272858 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":335,"threads_started":1}
I20260810 06:38:25.278249 13797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42eb8185bf954e68aba245a3a0fbe47f" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40871 } }
I20260810 06:38:25.278596 13797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:25.278689 13797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42eb8185bf954e68aba245a3a0fbe47f, State: Initialized, Role: FOLLOWER
I20260810 06:38:25.279192 13797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42eb8185bf954e68aba245a3a0fbe47f" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40871 } }
I20260810 06:38:25.279397 13797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:38:25.279520 13797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:38:25.279672 13797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:25.281651 13797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42eb8185bf954e68aba245a3a0fbe47f" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40871 } }
I20260810 06:38:25.282124 13797 leader_election.cc:302] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42eb8185bf954e68aba245a3a0fbe47f; no voters:
I20260810 06:38:25.282799 13797 leader_election.cc:288] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:38:25.282898 13804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:25.283484 13804 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [term 1 LEADER]: Becoming Leader. State: Replica: 42eb8185bf954e68aba245a3a0fbe47f, State: Running, Role: LEADER
I20260810 06:38:25.284157 13804 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42eb8185bf954e68aba245a3a0fbe47f" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40871 } }
I20260810 06:38:25.284885 13797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:25.286840 13805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42eb8185bf954e68aba245a3a0fbe47f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42eb8185bf954e68aba245a3a0fbe47f" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40871 } } }
I20260810 06:38:25.287228 13805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:25.287494 13808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42eb8185bf954e68aba245a3a0fbe47f. Latest consensus state: current_term: 1 leader_uuid: "42eb8185bf954e68aba245a3a0fbe47f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42eb8185bf954e68aba245a3a0fbe47f" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40871 } } }
I20260810 06:38:25.287824 13808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:25.289806 13814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:38:25.294653 13814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:38:25.302642 13814 catalog_manager.cc:1347] Generated new cluster ID: 87387e5c207f44afaff8e07b79168cbb
I20260810 06:38:25.302750 13814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:38:25.314049 13814 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:38:25.315779 13814 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:38:25.323257 13814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f: Generated new TSK 0
I20260810 06:38:25.324010 13814 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
I20260810 06:38:25.385027 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40871
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:25.702752 13840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:25.703018 13840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:25.703159 13840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:25.710378 13840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:25.710474 13840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:25.710616 13840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:38:25.720255 13840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:25.721413 13840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:25.722698 13840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:25.734956 13850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:25.735015 13855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:25.735087 13853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:25.736133 13840 server_base.cc:1034] running on GCE node
I20260810 06:38:25.736696 13840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:25.737547 13840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:25.738731 13840 hybrid_clock.cc:648] HybridClock initialized: now 1786343905738691 us; error 29 us; skew 500 ppm
I20260810 06:38:25.739012 13840 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:25 dist-test-slave-4tdj krb5kdc[13666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343905, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:25.741978 13840 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:38:25.743357 13840 webserver.cc:460] Webserver started at http://127.31.59.193:38703/ using document root <none> and password file <none>
I20260810 06:38:25.743971 13840 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:25.744093 13840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:25.744372 13840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:25.747148 13840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "468db36578aa4dadbdb808bac1b3b559"
format_stamp: "Formatted at 2026-08-10 06:38:25 on dist-test-slave-4tdj"
server_key: "43b940de5d381753f1aa6e88a73c2df0"
server_key_iv: "bd3497dd6e19e1ed5ed1c387e141f127"
server_key_version: "encryptionkey@0"
I20260810 06:38:25.747875 13840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "468db36578aa4dadbdb808bac1b3b559"
format_stamp: "Formatted at 2026-08-10 06:38:25 on dist-test-slave-4tdj"
server_key: "43b940de5d381753f1aa6e88a73c2df0"
server_key_iv: "bd3497dd6e19e1ed5ed1c387e141f127"
server_key_version: "encryptionkey@0"
I20260810 06:38:25.752712 13840 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:38:25.755856 13868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:25.756990 13840 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:38:25.757151 13840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "468db36578aa4dadbdb808bac1b3b559"
format_stamp: "Formatted at 2026-08-10 06:38:25 on dist-test-slave-4tdj"
server_key: "43b940de5d381753f1aa6e88a73c2df0"
server_key_iv: "bd3497dd6e19e1ed5ed1c387e141f127"
server_key_version: "encryptionkey@0"
I20260810 06:38:25.757328 13840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:25.770493 13840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:25.774274 13840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:25.774585 13840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:25.775409 13840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:25.776609 13840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:25.776686 13840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:25.776800 13840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:25.776857 13840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:25.806699 13840 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:37725
I20260810 06:38:25.806774 14065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:37725 every 8 connection(s)
I20260810 06:38:25.807866 13840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:38:25.816385 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 13840
I20260810 06:38:25.816577 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:38:25.817170 31983 external_mini_cluster.cc:1428] Setting key 69936af477123d79db8044a28d1607da
Aug 10 06:38:25 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343905, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:38:25.827451 14069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:25.810179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:47225 (local address 127.31.59.254:40871)
0810 06:38:25.810625 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:25.810629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:25.811473 (+ 844us) server_negotiation.cc:407] Connection header received
0810 06:38:25.813366 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:25.813372 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:25.813448 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:25.813554 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:25.814999 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:25.815756 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:25.817348 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:25.817894 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:25.818034 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:25.821626 (+ 3592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:25.821632 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:25.821636 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:25.821670 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:25.824595 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:25.825266 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:25.825271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:25.825274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:25.825409 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:25.825778 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:25.825782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:25.825785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:25.826123 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:25.826263 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:25.826922 (+ 659us) server_negotiation.cc:299] Negotiation successful
0810 06:38:25.827129 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":228,"threads_started":1}
I20260810 06:38:25.828131 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:25.810643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.193:47225)
0810 06:38:25.811316 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:25.811356 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:25.813116 (+ 1760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:25.813773 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:25.813780 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:25.814266 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:25.814835 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:25.814853 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:25.815901 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:25.815905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:25.817205 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:25.817218 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:25.818107 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:25.818112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:25.818392 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:25.819111 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:25.819140 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:25.821406 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:25.824741 (+ 3335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:25.824748 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:25.824758 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:25.825150 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:25.825493 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:25.825496 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:25.825498 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:25.825674 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:25.826237 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:25.826242 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:25.826586 (+ 344us) client_negotiation.cc:769] Sending connection context
0810 06:38:25.826795 (+ 209us) client_negotiation.cc:240] Negotiation successful
0810 06:38:25.827141 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":262,"threads_started":1}
I20260810 06:38:25.829492 14067 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40871
I20260810 06:38:25.829772 14067 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:25.830466 14067 heartbeater.cc:511] Master 127.31.59.254:40871 requested a full tablet report, sending...
I20260810 06:38:25.832648 13738 ts_manager.cc:194] Registered new tserver with Master: 468db36578aa4dadbdb808bac1b3b559 (127.31.59.193:37725)
I20260810 06:38:25.834517 13738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:47225
WARNING: no policy specified for kudu/127.31.59.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
I20260810 06:38:25.869052 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40871
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:26.185051 14075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:26.185349 14075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:26.185490 14075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:26.192754 14075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:26.192850 14075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:26.192999 14075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:38:26.203014 14075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:26.204202 14075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:26.205540 14075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:26.218863 14086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:26.219149 14089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:26.219391 14084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:26.220005 14075 server_base.cc:1034] running on GCE node
I20260810 06:38:26.220785 14075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:26.221940 14075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:26.224736 14075 hybrid_clock.cc:648] HybridClock initialized: now 1786343906224648 us; error 71 us; skew 500 ppm
I20260810 06:38:26.225087 14075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:26 dist-test-slave-4tdj krb5kdc[13666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343906, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:26.228686 14075 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:38:26.230103 14075 webserver.cc:460] Webserver started at http://127.31.59.194:41043/ using document root <none> and password file <none>
I20260810 06:38:26.230803 14075 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:26.230926 14075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:26.231222 14075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:26.234129 14075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "5e30134b3ed4433aacfeede704781b05"
format_stamp: "Formatted at 2026-08-10 06:38:26 on dist-test-slave-4tdj"
server_key: "792ace54524f83bc2f4fc7c6d4a41429"
server_key_iv: "c13293cf1e4ab29b2f911a97d2232760"
server_key_version: "encryptionkey@0"
I20260810 06:38:26.234961 14075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "5e30134b3ed4433aacfeede704781b05"
format_stamp: "Formatted at 2026-08-10 06:38:26 on dist-test-slave-4tdj"
server_key: "792ace54524f83bc2f4fc7c6d4a41429"
server_key_iv: "c13293cf1e4ab29b2f911a97d2232760"
server_key_version: "encryptionkey@0"
I20260810 06:38:26.240163 14075 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:38:26.243645 14103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:26.245067 14075 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:38:26.245247 14075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "5e30134b3ed4433aacfeede704781b05"
format_stamp: "Formatted at 2026-08-10 06:38:26 on dist-test-slave-4tdj"
server_key: "792ace54524f83bc2f4fc7c6d4a41429"
server_key_iv: "c13293cf1e4ab29b2f911a97d2232760"
server_key_version: "encryptionkey@0"
I20260810 06:38:26.245460 14075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:26.274644 14075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:26.278659 14075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:26.278913 14075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:26.279834 14075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:26.281136 14075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:26.281209 14075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:26.281324 14075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:26.281375 14075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:26.320087 14075 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:39365
I20260810 06:38:26.320168 14310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:39365 every 8 connection(s)
I20260810 06:38:26.321812 14075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:38:26.328640 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 14075
I20260810 06:38:26.328835 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:38:26.329377 31983 external_mini_cluster.cc:1428] Setting key 5300e47e7865a9960565edecfe8e3e03
Aug 10 06:38:26 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343906, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:38:26.345348 14069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:26.324792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:49269 (local address 127.31.59.254:40871)
0810 06:38:26.325000 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:26.325006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:26.326324 (+ 1318us) server_negotiation.cc:407] Connection header received
0810 06:38:26.329086 (+ 2762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:26.329092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:26.329177 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:26.329289 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:26.331417 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:26.332421 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:26.334898 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:26.335408 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:26.335523 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:26.339611 (+ 4088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:26.339616 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:26.339621 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:26.339659 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:26.342375 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:26.343168 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:26.343171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:26.343174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:26.343291 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:26.343717 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:26.343720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:26.343722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:26.343968 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:26.344128 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:26.344784 (+ 656us) server_negotiation.cc:299] Negotiation successful
0810 06:38:26.345002 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:38:26.346243 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:26.325261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.194:49269)
0810 06:38:26.326108 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:26.326171 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:26.328775 (+ 2604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:26.329572 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:26.329585 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:26.330360 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:26.331236 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:26.331265 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:26.332627 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:26.332633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:26.334743 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:26.334760 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:26.335658 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:26.335666 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:26.335975 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:26.336744 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:26.336775 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:26.339383 (+ 2608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:26.342503 (+ 3120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:26.342512 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:26.342527 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:26.343067 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:26.343380 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:26.343384 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:26.343388 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:26.343628 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:26.344112 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:26.344118 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:26.344427 (+ 309us) client_negotiation.cc:769] Sending connection context
0810 06:38:26.344656 (+ 229us) client_negotiation.cc:240] Negotiation successful
0810 06:38:26.345133 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":307,"threads_started":1}
I20260810 06:38:26.347782 14313 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40871
I20260810 06:38:26.348047 14313 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:26.348765 14313 heartbeater.cc:511] Master 127.31.59.254:40871 requested a full tablet report, sending...
I20260810 06:38:26.350099 13738 ts_manager.cc:194] Registered new tserver with Master: 5e30134b3ed4433aacfeede704781b05 (127.31.59.194:39365)
I20260810 06:38:26.350965 13738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:49269
WARNING: no policy specified for kudu/127.31.59.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
I20260810 06:38:26.381258 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40871
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:26.696252 14321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:26.696494 14321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:26.696581 14321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:26.703420 14321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:26.703511 14321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:26.703619 14321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:38:26.712679 14321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:26.713821 14321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:26.715046 14321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:26.727602 14331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:26.727640 14332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:26.727774 14335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:26.728583 14321 server_base.cc:1034] running on GCE node
I20260810 06:38:26.729141 14321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:26.730005 14321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:26.731205 14321 hybrid_clock.cc:648] HybridClock initialized: now 1786343906731144 us; error 54 us; skew 500 ppm
I20260810 06:38:26.731489 14321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:26 dist-test-slave-4tdj krb5kdc[13666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343906, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:26.738682 14321 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:38:26.740209 14321 webserver.cc:460] Webserver started at http://127.31.59.195:40577/ using document root <none> and password file <none>
I20260810 06:38:26.740957 14321 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:26.741086 14321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:26.741369 14321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:26.744346 14321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "35567cd8a7b548d7afaf542736ce6872"
format_stamp: "Formatted at 2026-08-10 06:38:26 on dist-test-slave-4tdj"
server_key: "7943ad983003e063b829fd61c56fc32d"
server_key_iv: "99fb6f25bf6f8246ffad09b4df7d426b"
server_key_version: "encryptionkey@0"
I20260810 06:38:26.745187 14321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "35567cd8a7b548d7afaf542736ce6872"
format_stamp: "Formatted at 2026-08-10 06:38:26 on dist-test-slave-4tdj"
server_key: "7943ad983003e063b829fd61c56fc32d"
server_key_iv: "99fb6f25bf6f8246ffad09b4df7d426b"
server_key_version: "encryptionkey@0"
I20260810 06:38:26.750491 14321 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 06:38:26.753839 14347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:26.755111 14321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:38:26.755255 14321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "35567cd8a7b548d7afaf542736ce6872"
format_stamp: "Formatted at 2026-08-10 06:38:26 on dist-test-slave-4tdj"
server_key: "7943ad983003e063b829fd61c56fc32d"
server_key_iv: "99fb6f25bf6f8246ffad09b4df7d426b"
server_key_version: "encryptionkey@0"
I20260810 06:38:26.755419 14321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:26.778813 14321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:26.783221 14321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:26.783460 14321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:26.784318 14321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:26.785590 14321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:26.785668 14321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:26.785763 14321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:26.785806 14321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:26.816033 14321 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:39153
I20260810 06:38:26.816082 14548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:39153 every 8 connection(s)
I20260810 06:38:26.817194 14321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:38:26.821602 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 14321
I20260810 06:38:26.821779 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:38:26.822191 31983 external_mini_cluster.cc:1428] Setting key 536987b21a29ca499203d74bef45e907
Aug 10 06:38:26 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343906, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:38:26.835422 14069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:26.819506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:58069 (local address 127.31.59.254:40871)
0810 06:38:26.819692 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:26.819697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:26.820814 (+ 1117us) server_negotiation.cc:407] Connection header received
0810 06:38:26.822700 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:26.822706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:26.822785 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:26.822871 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:26.824441 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:26.825235 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:26.826760 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:26.827156 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:26.827270 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:26.830399 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:26.830406 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:26.830410 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:26.830441 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:26.832630 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:26.833267 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:26.833272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:26.833274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:26.833396 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:26.833789 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:26.833793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:26.833795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:26.834074 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:26.834214 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:26.834877 (+ 663us) server_negotiation.cc:299] Negotiation successful
0810 06:38:26.835088 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:38:26.836371 14553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:26.819972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.195:58069)
0810 06:38:26.820645 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:26.820700 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:26.822413 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:26.823105 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:26.823114 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:26.823732 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:26.824273 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:26.824291 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:26.825377 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:26.825381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:26.826616 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:26.826628 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:26.827370 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:26.827378 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:26.827597 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:26.828203 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:26.828224 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:26.830193 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:26.832754 (+ 2561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:26.832762 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:26.832777 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:26.833150 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:26.833496 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:26.833501 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:26.833504 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:26.833670 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:26.834200 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:26.834207 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:26.834534 (+ 327us) client_negotiation.cc:769] Sending connection context
0810 06:38:26.834759 (+ 225us) client_negotiation.cc:240] Negotiation successful
0810 06:38:26.835219 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":258,"threads_started":1}
I20260810 06:38:26.837383 14067 heartbeater.cc:503] Master 127.31.59.254:40871 was elected leader, sending a full tablet report...
I20260810 06:38:26.837697 14549 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40871
I20260810 06:38:26.837965 14549 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:26.838835 14549 heartbeater.cc:511] Master 127.31.59.254:40871 requested a full tablet report, sending...
I20260810 06:38:26.840094 13737 ts_manager.cc:194] Registered new tserver with Master: 35567cd8a7b548d7afaf542736ce6872 (127.31.59.195:39153)
I20260810 06:38:26.840858 13737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:58069
I20260810 06:38:26.849136 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.193.keytab.
Entry for principal oryx/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.193.keytab.
WARNING: no policy specified for oryx/127.31.59.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.194.keytab.
Entry for principal oryx/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.194.keytab.
WARNING: no policy specified for oryx/127.31.59.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.195.keytab.
Entry for principal oryx/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.195.keytab.
I20260810 06:38:26.981397 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 13840
Aug 10 06:38:26 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:38:26.997591 14569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:26.982637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41866 (local address 127.31.59.193:37725)
0810 06:38:26.983354 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:26.983363 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:26.983395 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:38:26.983471 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:26.983479 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:26.983768 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:26.983917 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:26.985379 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:26.986483 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:26.987725 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:26.988282 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:26.988397 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:26.992386 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:26.992397 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:26.992409 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:26.992446 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:26.994993 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:26.995612 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:26.995619 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:26.995621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:26.995746 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:26.996095 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:26.996098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:26.996100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:26.996428 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:26.996708 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:26.997001 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 06:38:26.997264 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":366,"threads_started":1}
I20260810 06:38:26.998888 14056 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:27.080681 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 13840
I20260810 06:38:27.100968 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 14075
Aug 10 06:38:27 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.194@KRBTEST.COM
I20260810 06:38:27.117194 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:27.101861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53320 (local address 127.31.59.194:39365)
0810 06:38:27.102386 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:27.102394 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:27.102421 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:38:27.102482 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:27.102490 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:27.102699 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:27.102824 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:27.104022 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:27.105045 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:27.106283 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:27.106943 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:27.107062 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:27.111095 (+ 4033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:27.111107 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:27.111121 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:27.111169 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:27.114066 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:27.114922 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:27.114930 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:27.114932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:27.115085 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:27.115473 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:27.115480 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:27.115482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:27.115854 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:27.116191 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:27.116511 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 06:38:27.116793 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":251,"threads_started":1}
I20260810 06:38:27.118499 14303 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:27.199409 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 14075
I20260810 06:38:27.223033 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 14321
Aug 10 06:38:27 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.195@KRBTEST.COM
I20260810 06:38:27.239579 14575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:27.224065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52520 (local address 127.31.59.195:39153)
0810 06:38:27.224750 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:27.224758 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:27.224790 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:38:27.224855 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:27.224861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:27.225088 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:27.225263 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:27.226834 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:27.227899 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:27.229223 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:27.229811 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:27.229954 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:27.234337 (+ 4383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:27.234351 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:27.234365 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:27.234399 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:27.236864 (+ 2465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:27.237475 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:27.237483 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:27.237485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:27.237617 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:27.237972 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:27.237976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:27.237985 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:27.238378 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:27.238703 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:27.238990 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 06:38:27.239252 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":322,"threads_started":1}
I20260810 06:38:27.240759 14541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:27.322077 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 14321
I20260810 06:38:27.340070 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 13684
I20260810 06:38:27.341286 13789 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:27.443881 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 13684
I20260810 06:38:27.466179 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40871
--webserver_interface=127.31.59.254
--webserver_port=39337
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:40871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:27.783849 14580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:27.784114 14580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:27.784237 14580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:27.792631 14580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:27.792793 14580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:27.792863 14580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:27.792932 14580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:27.792985 14580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:27.803582 14580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:40871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=39337
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:27.804806 14580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:27.806100 14580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:27.817386 14592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:27.817489 14597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:27.817551 14591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:27.818400 14580 server_base.cc:1034] running on GCE node
I20260810 06:38:27.819183 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:27.820499 14580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:27.821722 14580 hybrid_clock.cc:648] HybridClock initialized: now 1786343907821651 us; error 48 us; skew 500 ppm
I20260810 06:38:27.822075 14580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:27 dist-test-slave-4tdj krb5kdc[13666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343907, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:27.825199 14580 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:38:27.826771 14580 webserver.cc:460] Webserver started at http://127.31.59.254:39337/ using document root <none> and password file <none>
I20260810 06:38:27.827423 14580 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:27.827571 14580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:27.832937 14580 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260810 06:38:27.835739 14607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:27.836992 14580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:38:27.837165 14580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "42eb8185bf954e68aba245a3a0fbe47f"
format_stamp: "Formatted at 2026-08-10 06:38:25 on dist-test-slave-4tdj"
server_key: "d61c29fe021d6e060029f6533391deec"
server_key_iv: "580b61ecc7af755ece5ecfa08d111bf9"
server_key_version: "encryptionkey@0"
I20260810 06:38:27.838107 14580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:27.864774 14580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:27.868966 14580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:27.869210 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:27.886567 14580 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:40871
I20260810 06:38:27.886543 14697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:40871 every 8 connection(s)
I20260810 06:38:27.888010 14580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:38:27.894626 14698 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:38:27.897516 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 14580
I20260810 06:38:27.897989 14698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f: Bootstrap starting.
I20260810 06:38:27.898505 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:37725
--local_ip_for_outbound_sockets=127.31.59.193
--tserver_master_addrs=127.31.59.254:40871
--webserver_port=38703
--webserver_interface=127.31.59.193
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.59.193
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:38:27.919078 14698 log.cc:824] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:27.927858 14698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:38:27.928341 14698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f: Bootstrap complete.
I20260810 06:38:27.937394 14698 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42eb8185bf954e68aba245a3a0fbe47f" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40871 } }
I20260810 06:38:27.938350 14698 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42eb8185bf954e68aba245a3a0fbe47f, State: Initialized, Role: FOLLOWER
I20260810 06:38:27.938999 14698 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42eb8185bf954e68aba245a3a0fbe47f" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40871 } }
I20260810 06:38:27.939224 14698 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:38:27.939332 14698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:38:27.939492 14698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:38:27.943434 14698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42eb8185bf954e68aba245a3a0fbe47f" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40871 } }
I20260810 06:38:27.943889 14698 leader_election.cc:302] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42eb8185bf954e68aba245a3a0fbe47f; no voters:
I20260810 06:38:27.944451 14698 leader_election.cc:288] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:38:27.944634 14706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:38:27.945571 14706 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [term 2 LEADER]: Becoming Leader. State: Replica: 42eb8185bf954e68aba245a3a0fbe47f, State: Running, Role: LEADER
I20260810 06:38:27.946199 14706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42eb8185bf954e68aba245a3a0fbe47f" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40871 } }
I20260810 06:38:27.946404 14698 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:27.951915 14708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "42eb8185bf954e68aba245a3a0fbe47f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42eb8185bf954e68aba245a3a0fbe47f" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40871 } } }
I20260810 06:38:27.952366 14709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42eb8185bf954e68aba245a3a0fbe47f. Latest consensus state: current_term: 2 leader_uuid: "42eb8185bf954e68aba245a3a0fbe47f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42eb8185bf954e68aba245a3a0fbe47f" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40871 } } }
I20260810 06:38:27.952679 14709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:27.952378 14708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:27.957267 14723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:38:27.963191 14723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:38:27.964635 14723 catalog_manager.cc:1259] Loaded cluster ID: 87387e5c207f44afaff8e07b79168cbb
I20260810 06:38:27.964781 14723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:38:27.968688 14723 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:38:27.970127 14723 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 42eb8185bf954e68aba245a3a0fbe47f: Loaded TSK: 0
I20260810 06:38:27.971683 14723 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:38:28.232774 14702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:28.233055 14702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:28.233193 14702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:28.240375 14702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:28.240499 14702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:28.240654 14702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:38:28.250234 14702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.193.keytab
--principal=oryx/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:37725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=38703
--tserver_master_addrs=127.31.59.254:40871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:28.251518 14702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:28.252789 14702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:28.266578 14750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:28.266988 14752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:28.267144 14758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:28.267915 14702 server_base.cc:1034] running on GCE node
I20260810 06:38:28.268476 14702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:28.269340 14702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:28.270529 14702 hybrid_clock.cc:648] HybridClock initialized: now 1786343908270477 us; error 44 us; skew 500 ppm
I20260810 06:38:28.270814 14702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:28 dist-test-slave-4tdj krb5kdc[13666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343908, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:28.273897 14702 init.cc:377] Logged in from keytab as oryx/127.31.59.193@KRBTEST.COM (short username oryx)
I20260810 06:38:28.275319 14702 webserver.cc:460] Webserver started at http://127.31.59.193:38703/ using document root <none> and password file <none>
I20260810 06:38:28.275949 14702 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:28.276078 14702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:28.281272 14702 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 06:38:28.283946 14774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:28.285264 14702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:38:28.285418 14702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "468db36578aa4dadbdb808bac1b3b559"
format_stamp: "Formatted at 2026-08-10 06:38:25 on dist-test-slave-4tdj"
server_key: "43b940de5d381753f1aa6e88a73c2df0"
server_key_iv: "bd3497dd6e19e1ed5ed1c387e141f127"
server_key_version: "encryptionkey@0"
I20260810 06:38:28.286352 14702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:28.303000 14702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:28.307004 14702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:28.307260 14702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:28.308125 14702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:28.309376 14702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:28.309474 14702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:28.309582 14702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:28.309652 14702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:28.339686 14702 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:37725
I20260810 06:38:28.339752 14993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:37725 every 8 connection(s)
I20260810 06:38:28.341114 14702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:38:28.344476 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 14702
I20260810 06:38:28.345227 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:39365
--local_ip_for_outbound_sockets=127.31.59.194
--tserver_master_addrs=127.31.59.254:40871
--webserver_port=41043
--webserver_interface=127.31.59.194
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.59.194
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:38:28 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:28 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:38:28.671552 15000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:28.671828 15000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:28.671960 15000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:28.678930 15000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:28.679035 15000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:28.679175 15000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:38:28.688318 15000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.194.keytab
--principal=oryx/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:39365
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=41043
--tserver_master_addrs=127.31.59.254:40871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:28.689599 15000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:28.691032 15000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:28.706056 15012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:28.706363 15016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:28.706061 15014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:28.707073 15000 server_base.cc:1034] running on GCE node
I20260810 06:38:28.707679 15000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:28.708578 15000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:28.709781 15000 hybrid_clock.cc:648] HybridClock initialized: now 1786343908709720 us; error 57 us; skew 500 ppm
I20260810 06:38:28.710080 15000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:28 dist-test-slave-4tdj krb5kdc[13666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343908, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:28.713189 15000 init.cc:377] Logged in from keytab as oryx/127.31.59.194@KRBTEST.COM (short username oryx)
I20260810 06:38:28.714705 15000 webserver.cc:460] Webserver started at http://127.31.59.194:41043/ using document root <none> and password file <none>
I20260810 06:38:28.715354 15000 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:28.715489 15000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:28.720767 15000 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.001s
I20260810 06:38:28.723373 15030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:28.724673 15000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:38:28.724817 15000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "5e30134b3ed4433aacfeede704781b05"
format_stamp: "Formatted at 2026-08-10 06:38:26 on dist-test-slave-4tdj"
server_key: "792ace54524f83bc2f4fc7c6d4a41429"
server_key_iv: "c13293cf1e4ab29b2f911a97d2232760"
server_key_version: "encryptionkey@0"
I20260810 06:38:28.725731 15000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:28.752197 15000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:28.756428 15000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:28.756691 15000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:28.757704 15000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:28.759083 15000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:28.759184 15000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:28.759287 15000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:28.759363 15000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:28.802898 15000 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:39365
I20260810 06:38:28.802953 15233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:39365 every 8 connection(s)
I20260810 06:38:28.804334 15000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:38:28.810499 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 15000
I20260810 06:38:28.811044 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:39153
--local_ip_for_outbound_sockets=127.31.59.195
--tserver_master_addrs=127.31.59.254:40871
--webserver_port=40577
--webserver_interface=127.31.59.195
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.59.195
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:38:28 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:28 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:38:29.341483 15244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:29.341917 15244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:29.342073 15244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:29.353556 15244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:29.353682 15244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:29.353912 15244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:38:29.368209 15244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/krb5kdc/oryx_127.31.59.195.keytab
--principal=oryx/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:39153
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=40577
--tserver_master_addrs=127.31.59.254:40871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:29.369769 15244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:29.371485 15244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:29.388667 15258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:29.388959 15255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:29.389189 15262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:29.398483 15244 server_base.cc:1034] running on GCE node
I20260810 06:38:29.399137 15244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:29.400249 15244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:29.401500 15244 hybrid_clock.cc:648] HybridClock initialized: now 1786343909401413 us; error 71 us; skew 500 ppm
I20260810 06:38:29.401829 15244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:29 dist-test-slave-4tdj krb5kdc[13666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343909, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:29.405719 15244 init.cc:377] Logged in from keytab as oryx/127.31.59.195@KRBTEST.COM (short username oryx)
I20260810 06:38:29.407539 15244 webserver.cc:460] Webserver started at http://127.31.59.195:40577/ using document root <none> and password file <none>
I20260810 06:38:29.408243 15244 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:29.408375 15244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:29.413627 15244 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260810 06:38:29.416429 15271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:29.417862 15244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:38:29.418016 15244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "35567cd8a7b548d7afaf542736ce6872"
format_stamp: "Formatted at 2026-08-10 06:38:26 on dist-test-slave-4tdj"
server_key: "7943ad983003e063b829fd61c56fc32d"
server_key_iv: "99fb6f25bf6f8246ffad09b4df7d426b"
server_key_version: "encryptionkey@0"
I20260810 06:38:29.418977 15244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:29.462263 15244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
W20260810 06:38:29.466005 14999 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.110s user 0.000s sys 0.012s
I20260810 06:38:29.467242 15244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:29.467578 15244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:29.468020 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:28.344072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.193:35835)
0810 06:38:28.345427 (+ 1355us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:28.345476 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:28.347691 (+ 2215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:28.349038 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:28.349047 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:28.349710 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:28.350443 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:28.350467 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:28.352021 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:28.352026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:28.353727 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:28.353741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:28.354755 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:28.354762 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:28.355017 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:28.355647 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:28.355668 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:29.466883 (+1111215us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1035,"thread_start_us":835,"threads_started":1}
W20260810 06:38:29.468457 14997 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:28.343969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:35835 (local address 127.31.59.254:40871)
0810 06:38:28.344738 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:28.344753 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:28.345589 (+ 836us) server_negotiation.cc:407] Connection header received
0810 06:38:28.348228 (+ 2639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:28.348242 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:28.348561 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:28.348838 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:28.350622 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:28.351846 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:28.353842 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:28.354529 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:28.354655 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:29.467443 (+1112788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.193:35835: BlockingRecv error: recv got EOF from 127.31.59.193:35835 (error 108)
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":263,"threads_started":1}
W20260810 06:38:29.469092 14995 heartbeater.cc:644] Failed to heartbeat to 127.31.59.254:40871 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.59.254:40871: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
I20260810 06:38:29.469985 15244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:29.472005 15244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:29.472182 15244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 06:38:29.472312 15244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:29.472393 15244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:38:29 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:29 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:29.478613 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:29.469704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.193:35385)
0810 06:38:29.469970 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:29.469989 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:29.470108 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:29.470473 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:29.470478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:29.470830 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:29.470984 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:29.470993 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.472138 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.472142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:29.473313 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:29.473326 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.474002 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.474009 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:29.474174 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:29.474677 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:29.474694 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:29.478360 (+ 3666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150}
W20260810 06:38:29.478793 14997 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:29.469835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:35385 (local address 127.31.59.254:40871)
0810 06:38:29.469981 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:29.469987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:29.470061 (+ 74us) server_negotiation.cc:407] Connection header received
0810 06:38:29.470245 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:29.470250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:29.470346 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:29.470433 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:29.471124 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.471990 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.473446 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.473833 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.473938 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:29.478542 (+ 4604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.193:35385: BlockingRecv error: recv got EOF from 127.31.59.193:35385 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 06:38:29 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:29 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:38:29.487139 14997 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:29.479442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:34837 (local address 127.31.59.254:40871)
0810 06:38:29.479582 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:29.479588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:29.479649 (+ 61us) server_negotiation.cc:407] Connection header received
0810 06:38:29.479807 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:29.479812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:29.479873 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:29.479946 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:29.480571 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.481247 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.482511 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.482942 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.483049 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:29.486911 (+ 3862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.193:34837: BlockingRecv error: recv got EOF from 127.31.59.193:34837 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:38:29.487581 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:29.479370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.193:34837)
0810 06:38:29.479580 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:29.479597 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:29.479695 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:29.479996 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:29.480001 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:29.480298 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:29.480443 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:29.480451 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.481376 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.481382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:29.482341 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:29.482356 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.483093 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.483099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:29.483267 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:29.483737 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:29.483753 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:29.487332 (+ 3579us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116}
W20260810 06:38:29.487936 14995 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:38:29.517899 15244 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:39153
I20260810 06:38:29.517910 15461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:39153 every 8 connection(s)
I20260810 06:38:29.520028 15244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:38:29.529429 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 15244
Aug 10 06:38:29 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:29 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:29.552438 15466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:29.530352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35868 (local address 127.31.59.254:40871)
0810 06:38:29.534258 (+ 3906us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:29.534265 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:29.534994 (+ 729us) server_negotiation.cc:407] Connection header received
0810 06:38:29.535204 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:29.535211 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:29.535287 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:29.535399 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:29.537540 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.538722 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.540647 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.541323 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.541430 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:29.545034 (+ 3604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:29.545045 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:29.545067 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:29.545110 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:29.548433 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:29.549048 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:29.549055 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:29.549062 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:29.549235 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:29.549570 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:29.549574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:29.549576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:29.550111 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:29.550445 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:29.551683 (+ 1238us) server_negotiation.cc:299] Negotiation successful
0810 06:38:29.552154 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3667,"thread_start_us":3471,"threads_started":1}
W20260810 06:38:29.857754 15239 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.038s user 0.002s sys 0.011s
W20260810 06:38:29.859241 15237 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:28.807339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:33121 (local address 127.31.59.254:40871)
0810 06:38:28.807825 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:28.807832 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:28.808894 (+ 1062us) server_negotiation.cc:407] Connection header received
0810 06:38:28.811791 (+ 2897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:28.811797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:28.811880 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:28.811998 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:28.814172 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:28.815149 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:28.817597 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:28.818154 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:28.818272 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:29.858977 (+1040705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.194:33121: BlockingRecv error: recv got EOF from 127.31.59.194:33121 (error 108)
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":226,"threads_started":1}
I20260810 06:38:29.860344 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:28.807831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.194:33121)
0810 06:38:28.808712 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:28.808767 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:28.811459 (+ 2692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:28.812282 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:28.812296 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:28.813105 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:28.813949 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:28.813987 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:28.815295 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:28.815301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:28.817432 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:28.817450 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:28.818407 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:28.818415 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:28.818734 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:28.819538 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:28.819572 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:29.858862 (+1039290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":308,"threads_started":1}
W20260810 06:38:29.861338 15235 heartbeater.cc:644] Failed to heartbeat to 127.31.59.254:40871 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.59.254:40871: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Aug 10 06:38:29 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:29 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:38:29.870536 15237 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:29.862054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:48631 (local address 127.31.59.254:40871)
0810 06:38:29.862229 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:29.862235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:29.862252 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:38:29.862420 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:29.862426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:29.862503 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:29.862586 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:29.863311 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.864113 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.865368 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.865772 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.865879 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:29.870331 (+ 4452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.194:48631: BlockingRecv error: recv got EOF from 127.31.59.194:48631 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:38:29.871129 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:29.861966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.194:48631)
0810 06:38:29.862149 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:29.862170 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:29.862284 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:29.862648 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:29.862654 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:29.863010 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:29.863174 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:29.863183 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.864249 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.864255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:29.865232 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:29.865246 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.865939 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.865946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:29.866103 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:29.866696 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:29.866714 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:29.870807 (+ 4093us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
Aug 10 06:38:29 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:29 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:38:29.881556 15237 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:29.871963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:34071 (local address 127.31.59.254:40871)
0810 06:38:29.872113 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:29.872117 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:29.872214 (+ 97us) server_negotiation.cc:407] Connection header received
0810 06:38:29.872385 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:29.872389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:29.872454 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:29.872523 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:29.873369 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.874086 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.876083 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.876474 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.876570 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:29.881329 (+ 4759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.194:34071: BlockingRecv error: recv got EOF from 127.31.59.194:34071 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:38:29.881935 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:29.871756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.194:34071)
0810 06:38:29.872134 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:29.872157 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:29.872283 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:29.872582 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:29.872587 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:29.873017 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:29.873246 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:29.873258 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.874211 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.874217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:29.875937 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:29.875953 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.876619 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.876626 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:29.876826 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:29.877460 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:29.877495 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:29.881754 (+ 4259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 06:38:29.882269 15235 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 06:38:30 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:30 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:30.499189 15474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:30.488505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.193:51223)
0810 06:38:30.489248 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:30.489269 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:30.489393 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:30.489783 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:30.489788 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:30.490418 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:30.490624 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:30.490635 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:30.491701 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:30.491707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:30.493218 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:30.493236 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:30.494268 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:30.494276 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:30.494523 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:30.495192 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:30.495213 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:30.498807 (+ 3594us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":456,"threads_started":1}
W20260810 06:38:30.499305 15475 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:30.488592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:51223 (local address 127.31.59.254:40871)
0810 06:38:30.489502 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:30.489506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:30.489521 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:38:30.489564 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:30.489568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:30.489652 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:30.489738 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:30.490762 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:30.491537 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:30.493374 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:30.494037 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:30.494176 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:30.499046 (+ 4870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.193:51223: BlockingRecv error: recv got EOF from 127.31.59.193:51223 (error 108)
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":702,"threads_started":1}
W20260810 06:38:30.519954 15464 sasl_common.cc:323] Time spent calling sasl_client_start(): real 0.986s user 0.003s sys 0.010s
W20260810 06:38:30.521121 14997 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:29.522864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:43391 (local address 127.31.59.254:40871)
0810 06:38:29.523043 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:29.523049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:29.524756 (+ 1707us) server_negotiation.cc:407] Connection header received
0810 06:38:29.526601 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:29.526607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:29.526690 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:29.526775 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:29.528261 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.529018 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.531737 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.532440 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.532547 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:30.520863 (+988316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.195:43391: BlockingRecv error: recv got EOF from 127.31.59.195:43391 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:38:30.521721 15464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:29.523736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.195:43391)
0810 06:38:29.524550 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:29.524598 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:29.526333 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:29.527022 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:29.527031 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:29.527531 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:29.528093 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:29.528110 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.529223 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.529230 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:29.531508 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:29.531534 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:29.532960 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:29.532967 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:29.533294 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:29.534149 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:29.534181 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:30.520730 (+986549us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":325,"threads_started":1}
W20260810 06:38:30.522569 15462 heartbeater.cc:644] Failed to heartbeat to 127.31.59.254:40871 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.59.254:40871: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Aug 10 06:38:30 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:30 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:30.532136 15464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:30.523379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.195:60699)
0810 06:38:30.523626 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:30.523646 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:30.523761 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:30.524062 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:30.524068 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:30.524444 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:30.524615 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:30.524624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:30.525658 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:30.525664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:30.526673 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:30.526687 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:30.527393 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:30.527399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:30.527557 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:30.528083 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:30.528099 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:30.531804 (+ 3705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":70}
W20260810 06:38:30.532500 14997 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:30.523403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:60699 (local address 127.31.59.254:40871)
0810 06:38:30.523632 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:30.523638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:30.523695 (+ 57us) server_negotiation.cc:407] Connection header received
0810 06:38:30.523867 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:30.523873 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:30.523939 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:30.524032 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:30.524754 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:30.525525 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:30.526807 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:30.527226 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:30.527338 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:30.532186 (+ 4848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.195:60699: BlockingRecv error: recv got EOF from 127.31.59.195:60699 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
Aug 10 06:38:30 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:30 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:30.540200 15464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:30.532854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.195:54025)
0810 06:38:30.532984 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:30.533000 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:30.533097 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:30.533373 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:30.533378 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:30.533677 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:30.533819 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:30.533827 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:30.534854 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:30.534858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:30.535782 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:30.535794 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:30.536444 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:30.536450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:30.536616 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:30.537098 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:30.537114 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:30.539959 (+ 2845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 06:38:30.540359 14997 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:30.532960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:54025 (local address 127.31.59.254:40871)
0810 06:38:30.533122 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:30.533128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:30.533144 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:38:30.533187 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:30.533192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:30.533252 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:30.533327 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:30.533949 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:30.534726 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:30.535913 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:30.536307 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:30.536411 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:30.540159 (+ 3748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.195:54025: BlockingRecv error: recv got EOF from 127.31.59.195:54025 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 06:38:30.540488 15462 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 06:38:30 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:30 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:30.892599 15476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:30.882911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.194:36997)
0810 06:38:30.883508 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:30.883527 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:30.883654 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:30.883974 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:30.883978 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:30.884436 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:30.884626 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:30.884637 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:30.885564 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:30.885568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:30.886691 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:30.886707 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:30.887708 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:30.887715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:30.887851 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:30.888506 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:30.888526 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:30.892353 (+ 3827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":324,"threads_started":1}
W20260810 06:38:30.892879 14997 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:30.882984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:36997 (local address 127.31.59.254:40871)
0810 06:38:30.883197 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:30.883204 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:30.883639 (+ 435us) server_negotiation.cc:407] Connection header received
0810 06:38:30.883769 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:30.883773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:30.883858 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:30.883934 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:30.884744 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:30.885455 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:30.886827 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:30.887518 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:30.887633 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:30.892628 (+ 4995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.194:36997: BlockingRecv error: recv got EOF from 127.31.59.194:36997 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 10 06:38:31 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:31 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:31.509363 15477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:31.500110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.193:50813)
0810 06:38:31.500628 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:31.500644 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:31.500750 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:31.501137 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:31.501139 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:31.501559 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:31.501745 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:31.501755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:31.502922 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:31.502926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:31.503872 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:31.503886 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:31.504708 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:31.504715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:31.504889 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:31.505505 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:31.505531 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:31.509018 (+ 3487us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":261,"threads_started":1}
W20260810 06:38:31.509465 15480 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:31.500201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:50813 (local address 127.31.59.254:40871)
0810 06:38:31.500900 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:31.500904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:31.500919 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:38:31.500961 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:31.500964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:31.501021 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:31.501102 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:31.501905 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:31.502765 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:31.504025 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:31.504531 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:31.504634 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:31.509211 (+ 4577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.193:50813: BlockingRecv error: recv got EOF from 127.31.59.193:50813 (error 108)
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":506,"threads_started":1}
Aug 10 06:38:31 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:31 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:31.550071 15485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:31.541144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.195:41181)
0810 06:38:31.541689 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:31.541711 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:31.541832 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:31.542223 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:31.542229 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:31.542846 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:31.543024 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:31.543036 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:31.544112 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:31.544116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:31.545043 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:31.545055 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:31.545742 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:31.545747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:31.545862 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:31.546462 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:31.546478 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:31.549836 (+ 3358us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":308,"threads_started":1}
W20260810 06:38:31.550225 15480 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:31.541328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:41181 (local address 127.31.59.254:40871)
0810 06:38:31.541557 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:31.541563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:31.541795 (+ 232us) server_negotiation.cc:407] Connection header received
0810 06:38:31.541958 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:31.541977 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:31.542078 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:31.542165 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:31.543164 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:31.543957 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:31.545166 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:31.545600 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:31.545701 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:31.550026 (+ 4325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.195:41181: BlockingRecv error: recv got EOF from 127.31.59.195:41181 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 10 06:38:31 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:31 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:31.911152 15489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:31.893654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.194:50911)
0810 06:38:31.894130 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:31.894146 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:31.894307 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:31.894634 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:31.894639 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:31.895092 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:31.895274 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:31.895283 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:31.896310 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:31.896316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:31.897249 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:31.897261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:31.897978 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:31.897985 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:31.898158 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:31.907416 (+ 9258us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:31.907436 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:31.910808 (+ 3372us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":261,"threads_started":1}
W20260810 06:38:31.911268 15480 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:31.893713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:50911 (local address 127.31.59.254:40871)
0810 06:38:31.893894 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:31.893900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:31.894228 (+ 328us) server_negotiation.cc:407] Connection header received
0810 06:38:31.894422 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:31.894428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:31.894497 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:31.894582 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:31.895419 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:31.896166 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:31.897390 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:31.897826 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:31.897936 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:31.911025 (+ 13089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.194:50911: BlockingRecv error: recv got EOF from 127.31.59.194:50911 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 06:38:32 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:32 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:32.519184 15490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:32.510331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.193:41675)
0810 06:38:32.510874 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:32.510891 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:32.511002 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:32.511271 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:32.511274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:32.511637 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:32.511813 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:32.511821 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:32.512799 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:32.512802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:32.513819 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:32.513834 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:32.514700 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:32.514706 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:32.514846 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:32.515478 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:32.515497 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:32.518849 (+ 3352us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":295,"threads_started":1}
W20260810 06:38:32.519325 15491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:32.510489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:41675 (local address 127.31.59.254:40871)
0810 06:38:32.510961 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:32.510965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:32.510981 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:38:32.511095 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:32.511099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:32.511157 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:32.511244 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:32.511954 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:32.512699 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:32.513962 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:32.514522 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:32.514682 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:32.519057 (+ 4375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.193:41675: BlockingRecv error: recv got EOF from 127.31.59.193:41675 (error 108)
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":267,"threads_started":1}
Aug 10 06:38:32 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:32 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:32.559077 15492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:32.550891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.195:52299)
0810 06:38:32.551394 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:32.551418 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:32.551542 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:32.551801 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:32.551805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:32.552206 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:32.552390 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:32.552400 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:32.553332 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:32.553336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:32.554273 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:32.554286 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:32.554889 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:32.554893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:32.555021 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:32.555512 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:32.555532 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:32.558802 (+ 3270us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":295,"threads_started":1}
W20260810 06:38:32.559176 15491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:32.550948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:52299 (local address 127.31.59.254:40871)
0810 06:38:32.551105 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:32.551110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:32.551481 (+ 371us) server_negotiation.cc:407] Connection header received
0810 06:38:32.551632 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:32.551636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:32.551694 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:32.551769 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:32.552495 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:32.553225 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:32.554397 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:32.554779 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:32.554862 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:32.558967 (+ 4105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.195:52299: BlockingRecv error: recv got EOF from 127.31.59.195:52299 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 10 06:38:32 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:32 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:32.921224 15493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:32.912367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.194:42603)
0810 06:38:32.912904 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:32.912922 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:32.913068 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:32.913440 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:32.913447 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:32.913842 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:32.914034 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:32.914044 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:32.915049 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:32.915054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:32.916027 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:32.916041 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:32.916762 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:32.916768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:32.916903 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:32.917533 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:32.917553 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:32.920893 (+ 3340us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":282,"threads_started":1}
W20260810 06:38:32.921350 15491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:32.912413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:42603 (local address 127.31.59.254:40871)
0810 06:38:32.912609 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:32.912615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:32.913003 (+ 388us) server_negotiation.cc:407] Connection header received
0810 06:38:32.913194 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:32.913200 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:32.913306 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:32.913402 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:32.914153 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:32.914908 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:32.916159 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:32.916615 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:32.916705 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:32.921104 (+ 4399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.194:42603: BlockingRecv error: recv got EOF from 127.31.59.194:42603 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 10 06:38:33 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:33 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:33.529858 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:33.520742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.193:41831)
0810 06:38:33.521281 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:33.521301 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:33.521453 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:33.521848 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:33.521852 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:33.522240 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:33.522486 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:33.522496 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:33.523670 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:33.523675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:33.524750 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:33.524764 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:33.525479 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:33.525484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:33.525623 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:33.526169 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:33.526187 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:33.529610 (+ 3423us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":280,"threads_started":1}
W20260810 06:38:33.530081 15497 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:33.520805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:41831 (local address 127.31.59.254:40871)
0810 06:38:33.521262 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:33.521268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:33.521369 (+ 101us) server_negotiation.cc:407] Connection header received
0810 06:38:33.521601 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:33.521606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:33.521685 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:33.521785 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:33.522635 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:33.523541 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:33.524907 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:33.525326 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:33.525423 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:33.529801 (+ 4378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.193:41831: BlockingRecv error: recv got EOF from 127.31.59.193:41831 (error 108)
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":243,"threads_started":1}
Aug 10 06:38:33 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:33 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:33.568413 15500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:33.559967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.195:42391)
0810 06:38:33.560448 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:33.560468 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:33.560602 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:33.560914 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:33.560920 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:33.561311 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:33.561490 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:33.561499 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:33.562560 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:33.562566 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:33.563471 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:33.563483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:33.564124 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:33.564130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:33.564293 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:33.564852 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:33.564869 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:33.568097 (+ 3228us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":248,"threads_started":1}
W20260810 06:38:33.568497 15497 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:33.560064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:42391 (local address 127.31.59.254:40871)
0810 06:38:33.560317 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:33.560323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:33.560573 (+ 250us) server_negotiation.cc:407] Connection header received
0810 06:38:33.560713 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:33.560719 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:33.560785 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:33.560863 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:33.561639 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:33.562434 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:33.563610 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:33.563990 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:33.564096 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:33.568299 (+ 4203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.195:42391: BlockingRecv error: recv got EOF from 127.31.59.195:42391 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
Aug 10 06:38:33 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:33 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:33.931314 15501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:33.922355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.194:47947)
0810 06:38:33.922880 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:33.922895 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:33.923005 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:33.923316 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:33.923321 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:33.923840 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:33.924016 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:33.924026 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:33.924982 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:33.924986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:33.925992 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:33.926005 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:33.926826 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:33.926833 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:33.927004 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:33.927591 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:33.927609 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:33.931062 (+ 3453us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":302,"threads_started":1}
W20260810 06:38:33.931442 15497 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:33.922391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:47947 (local address 127.31.59.254:40871)
0810 06:38:33.922606 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:33.922614 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:33.922964 (+ 350us) server_negotiation.cc:407] Connection header received
0810 06:38:33.923106 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:33.923110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:33.923185 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:33.923260 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:33.924127 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:33.924875 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:33.926113 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:33.926691 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:33.926812 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:33.931227 (+ 4415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.194:47947: BlockingRecv error: recv got EOF from 127.31.59.194:47947 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 06:38:34 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:34 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:34.540094 15504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:34.530781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.193:56113)
0810 06:38:34.531279 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:34.531296 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:34.531397 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:34.531749 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:34.531757 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:34.532180 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:34.532342 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:34.532351 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:34.533581 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:34.533585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:34.534961 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:34.534975 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:34.535659 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:34.535663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:34.535790 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:34.536380 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:34.536399 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:34.539861 (+ 3462us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":253,"threads_started":1}
W20260810 06:38:34.540305 15505 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:34.530852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:56113 (local address 127.31.59.254:40871)
0810 06:38:34.531328 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:34.531332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:34.531349 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:38:34.531487 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:34.531490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:34.531562 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:34.531654 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:34.532523 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:34.533424 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:34.535086 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:34.535512 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:34.535609 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:34.540071 (+ 4462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.193:56113: BlockingRecv error: recv got EOF from 127.31.59.193:56113 (error 108)
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":244,"threads_started":1}
Aug 10 06:38:34 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:34 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:34.577836 15506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:34.569475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.195:34839)
0810 06:38:34.569970 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:34.569991 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:34.570127 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:34.570458 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:34.570463 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:34.570870 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:34.571049 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:34.571059 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:34.572058 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:34.572063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:34.573028 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:34.573042 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:34.573699 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:34.573705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:34.573868 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:34.574428 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:34.574445 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:34.577505 (+ 3060us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":273,"threads_started":1}
W20260810 06:38:34.577950 15505 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:34.569643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:34839 (local address 127.31.59.254:40871)
0810 06:38:34.569823 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:34.569829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:34.570083 (+ 254us) server_negotiation.cc:407] Connection header received
0810 06:38:34.570236 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:34.570242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:34.570324 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:34.570402 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:34.571192 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:34.571914 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:34.573169 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:34.573552 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:34.573666 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:34.577726 (+ 4060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.195:34839: BlockingRecv error: recv got EOF from 127.31.59.195:34839 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 10 06:38:34 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:34 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:34.940462 15507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:34.932148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.194:55473)
0810 06:38:34.932629 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:34.932645 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:34.932759 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:34.933056 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:34.933060 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:34.933441 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:34.933602 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:34.933611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:34.934612 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:34.934616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:34.935550 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:34.935563 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:34.936230 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:34.936234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:34.936360 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:34.936916 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:34.936934 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:34.940148 (+ 3214us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":247,"threads_started":1}
W20260810 06:38:34.940613 15505 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:34.932292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:55473 (local address 127.31.59.254:40871)
0810 06:38:34.932441 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:34.932446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:34.932706 (+ 260us) server_negotiation.cc:407] Connection header received
0810 06:38:34.932856 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:34.932860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:34.932933 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:34.933003 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:34.933734 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:34.934481 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:34.935683 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:34.936111 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:34.936203 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:34.940370 (+ 4167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.194:55473: BlockingRecv error: recv got EOF from 127.31.59.194:55473 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 06:38:35 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:35 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:35.550078 15511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:35.541069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.193:60459)
0810 06:38:35.541573 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:35.541592 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:35.541698 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:35.541968 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:35.541971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:35.542454 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:35.542644 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:35.542655 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:35.543646 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:35.543650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:35.544576 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:35.544588 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:35.545257 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:35.545262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:35.545385 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:35.545955 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:35.545976 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:35.549839 (+ 3863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":255,"threads_started":1}
W20260810 06:38:35.550325 15515 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:35.541114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:60459 (local address 127.31.59.254:40871)
0810 06:38:35.541673 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:35.541678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:35.541695 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:38:35.541791 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:35.541795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:35.541854 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:35.541940 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:35.542776 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:35.543538 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:35.544714 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:35.545112 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:35.545202 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:35.550052 (+ 4850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.193:60459: BlockingRecv error: recv got EOF from 127.31.59.193:60459 (error 108)
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":244,"threads_started":1}
Aug 10 06:38:35 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:35 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:35.587625 15517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:35.578801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.195:53537)
0810 06:38:35.579347 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:35.579368 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:35.579503 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:35.579851 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:35.579857 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:35.580279 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:35.580440 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:35.580450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:35.581497 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:35.581501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:35.582503 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:35.582517 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:35.583323 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:35.583329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:35.583489 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:35.584055 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:35.584072 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:35.587258 (+ 3186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":306,"threads_started":1}
W20260810 06:38:35.587656 15515 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:35.578907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:53537 (local address 127.31.59.254:40871)
0810 06:38:35.579147 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:35.579154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:35.579454 (+ 300us) server_negotiation.cc:407] Connection header received
0810 06:38:35.579639 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:35.579646 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:35.579718 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:35.579799 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:35.580584 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:35.581355 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:35.582640 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:35.583182 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:35.583292 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:35.587451 (+ 4159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.195:53537: BlockingRecv error: recv got EOF from 127.31.59.195:53537 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 10 06:38:35 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:35 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:35.950666 15520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:35.941558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.194:49953)
0810 06:38:35.942084 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:35.942102 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:35.942249 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:35.942659 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:35.942666 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:35.943135 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:35.943326 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:35.943337 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:35.944437 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:35.944442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:35.945452 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:35.945468 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:35.946220 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:35.946228 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:35.946435 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:35.947064 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:35.947103 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:35.950424 (+ 3321us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":278,"threads_started":1}
W20260810 06:38:35.950851 15515 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:35.941676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:49953 (local address 127.31.59.254:40871)
0810 06:38:35.941868 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:35.941876 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:35.942199 (+ 323us) server_negotiation.cc:407] Connection header received
0810 06:38:35.942391 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:35.942398 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:35.942509 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:35.942604 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:35.943514 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:35.944281 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:35.945609 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:35.946059 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:35.946181 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:35.950608 (+ 4427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.194:49953: BlockingRecv error: recv got EOF from 127.31.59.194:49953 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 06:38:36 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:36 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:36.560019 15521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:36.550976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.193:40153)
0810 06:38:36.551591 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:36.551609 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:36.551730 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:36.552044 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:36.552047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:36.552461 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:36.552661 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:36.552671 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:36.553675 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:36.553679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:36.554653 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:36.554665 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:36.555394 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:36.555398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:36.555555 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:36.556249 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:36.556272 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:36.559772 (+ 3500us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":273,"threads_started":1}
W20260810 06:38:36.560194 15522 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:36.551056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:40153 (local address 127.31.59.254:40871)
0810 06:38:36.551568 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:36.551572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:36.551679 (+ 107us) server_negotiation.cc:407] Connection header received
0810 06:38:36.551830 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:36.551834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:36.551908 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:36.551988 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:36.552798 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:36.553560 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:36.554795 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:36.555246 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:36.555368 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:36.559957 (+ 4589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.193:40153: BlockingRecv error: recv got EOF from 127.31.59.193:40153 (error 108)
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":261,"threads_started":1}
Aug 10 06:38:36 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:36 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:36.597193 15523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:36.588579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.195:51987)
0810 06:38:36.589088 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:36.589108 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:36.589220 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:36.589502 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:36.589505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:36.589903 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:36.590149 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:36.590165 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:36.591430 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:36.591435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:36.592369 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:36.592383 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:36.593087 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:36.593091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:36.593211 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:36.593733 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:36.593750 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:36.596859 (+ 3109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":255,"threads_started":1}
W20260810 06:38:36.597282 15522 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:36.588657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:51987 (local address 127.31.59.254:40871)
0810 06:38:36.588874 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:36.588881 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:36.589172 (+ 291us) server_negotiation.cc:407] Connection header received
0810 06:38:36.589304 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:36.589307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:36.589383 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:36.589467 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:36.590321 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:36.591293 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:36.592526 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:36.592957 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:36.593066 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:36.597078 (+ 4012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.195:51987: BlockingRecv error: recv got EOF from 127.31.59.195:51987 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 10 06:38:36 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:36 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:36.964474 15528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:36.951515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.194:57481)
0810 06:38:36.952133 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:36.952150 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:36.952282 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:36.952619 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:36.952625 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:36.953042 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:36.953224 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:36.953234 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:36.954232 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:36.954237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:36.955236 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:36.955250 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:36.955907 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:36.955915 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:36.956113 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:36.960676 (+ 4563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:36.960697 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:36.964127 (+ 3430us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":292,"threads_started":1}
W20260810 06:38:36.964617 15522 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:36.951619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:57481 (local address 127.31.59.254:40871)
0810 06:38:36.951864 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:36.951871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:36.952233 (+ 362us) server_negotiation.cc:407] Connection header received
0810 06:38:36.952407 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:36.952411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:36.952473 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:36.952552 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:36.953370 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:36.954094 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:36.955360 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:36.955760 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:36.955869 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:36.964356 (+ 8487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.194:57481: BlockingRecv error: recv got EOF from 127.31.59.194:57481 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
Aug 10 06:38:37 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:37 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:37.569530 15531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:37.560900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.193:54829)
0810 06:38:37.561408 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:37.561425 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:37.561536 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:37.561926 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:37.561929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:37.562339 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:37.562536 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:37.562546 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:37.563560 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:37.563564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:37.564482 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:37.564494 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:37.565149 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:37.565153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:37.565285 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:37.565878 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:37.565895 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:37.569141 (+ 3246us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":249,"threads_started":1}
W20260810 06:38:37.569998 15534 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:37.561004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:54829 (local address 127.31.59.254:40871)
0810 06:38:37.561688 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:37.561692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:37.561706 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:38:37.561747 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:37.561750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:37.561815 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:37.561898 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:37.562672 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:37.563426 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:37.564596 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:37.565019 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:37.565104 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:37.569387 (+ 4283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.193:54829: BlockingRecv error: recv got EOF from 127.31.59.193:54829 (error 108)
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":477,"threads_started":1}
Aug 10 06:38:37 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:37 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:37.606992 15535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:37.598056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.195:34555)
0810 06:38:37.598708 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:37.598728 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:37.598862 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:37.599189 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:37.599194 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:37.599583 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:37.599758 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:37.599768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:37.600748 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:37.600752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:37.601707 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:37.601720 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:37.602417 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:37.602423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:37.602588 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:37.603154 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:37.603172 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:37.606676 (+ 3504us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":290,"threads_started":1}
W20260810 06:38:37.607108 15534 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:37.598247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:34555 (local address 127.31.59.254:40871)
0810 06:38:37.598452 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:37.598458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:37.598809 (+ 351us) server_negotiation.cc:407] Connection header received
0810 06:38:37.598967 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:37.598972 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:37.599045 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:37.599125 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:37.599904 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:37.600616 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:37.601841 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:37.602257 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:37.602400 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:37.606869 (+ 4469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.195:34555: BlockingRecv error: recv got EOF from 127.31.59.195:34555 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 10 06:38:37 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:37 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:37.974397 15536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:37.965632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.194:39201)
0810 06:38:37.966202 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:37.966222 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:37.966379 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:37.966696 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:37.966702 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:37.967138 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:37.967307 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:37.967317 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:37.968484 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:37.968491 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:37.969421 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:37.969432 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:37.970116 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:37.970123 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:37.970279 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:37.970842 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:37.970859 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:37.974050 (+ 3191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":283,"threads_started":1}
W20260810 06:38:37.974550 15534 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:37.965778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:39201 (local address 127.31.59.254:40871)
0810 06:38:37.966018 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:37.966024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:37.966316 (+ 292us) server_negotiation.cc:407] Connection header received
0810 06:38:37.966494 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:37.966499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:37.966566 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:37.966645 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:37.967448 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:37.968341 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:37.969556 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:37.969943 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:37.970057 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:37.974277 (+ 4220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.194:39201: BlockingRecv error: recv got EOF from 127.31.59.194:39201 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 10 06:38:38 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:38 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:38.579000 15541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.570542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.193:57649)
0810 06:38:38.571030 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:38.571048 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:38.571157 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:38.571432 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:38.571436 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:38.571859 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:38.572005 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.572014 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.573183 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.573188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.574328 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.574342 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.574966 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.574971 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.575093 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.575645 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:38.575661 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:38.578783 (+ 3122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":241,"threads_started":1}
W20260810 06:38:38.579200 15544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:38.570620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:57649 (local address 127.31.59.254:40871)
0810 06:38:38.571099 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:38.571103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:38.571119 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:38:38.571251 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:38.571254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:38.571318 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:38.571403 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:38.572148 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.573027 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.574456 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.574856 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.574945 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.578956 (+ 4011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.193:57649: BlockingRecv error: recv got EOF from 127.31.59.193:57649 (error 108)
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":245,"threads_started":1}
Aug 10 06:38:38 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:38 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:38.615993 15545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.607951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.195:50131)
0810 06:38:38.608574 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:38.608593 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:38.608704 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:38.608956 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:38.608960 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:38.609353 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:38.609532 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.609541 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.610487 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.610491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.611425 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.611437 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.612101 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.612106 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.612214 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.612721 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:38.612737 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:38.615758 (+ 3021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":384,"threads_started":1}
W20260810 06:38:38.616127 15544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:38.608028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:50131 (local address 127.31.59.254:40871)
0810 06:38:38.608229 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:38.608235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:38.608668 (+ 433us) server_negotiation.cc:407] Connection header received
0810 06:38:38.608798 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:38.608802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:38.608862 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:38.608928 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:38.609638 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.610392 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.611548 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.611956 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.612043 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.615926 (+ 3883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.195:50131: BlockingRecv error: recv got EOF from 127.31.59.195:50131 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 10 06:38:38 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:38 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:38.984087 15546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.975303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.194:54477)
0810 06:38:38.975826 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:38.975842 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:38.975954 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:38.976307 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:38.976312 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:38.976713 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:38.976865 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.976875 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.977825 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.977829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.978965 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.978980 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.979897 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.979903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.980043 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.980598 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:38.980615 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:38.983858 (+ 3243us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":289,"threads_started":1}
W20260810 06:38:38.984311 15544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:38.975411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:54477 (local address 127.31.59.254:40871)
0810 06:38:38.975596 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:38.975601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:38.975916 (+ 315us) server_negotiation.cc:407] Connection header received
0810 06:38:38.976043 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:38.976047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:38.976152 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:38.976239 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:38.976987 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.977719 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.979126 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.979755 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.979857 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.984049 (+ 4192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.194:54477: BlockingRecv error: recv got EOF from 127.31.59.194:54477 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 10 06:38:39 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:39 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:39.587775 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:39.579821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.193:40579)
0810 06:38:39.580327 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:39.580344 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:39.580456 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:39.580722 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:39.580725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:39.581115 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:39.581283 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:39.581293 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:39.582258 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:39.582262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:39.583263 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:39.583276 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:39.583893 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:39.583897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:39.584008 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:39.584523 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:39.584540 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:39.587537 (+ 2997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":274,"threads_started":1}
W20260810 06:38:39.587934 15550 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:39.579895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:40579 (local address 127.31.59.254:40871)
0810 06:38:39.580346 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:39.580350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:39.580409 (+ 59us) server_negotiation.cc:407] Connection header received
0810 06:38:39.580539 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:39.580542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:39.580603 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:39.580688 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:39.581395 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:39.582147 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:39.583370 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:39.583785 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:39.583873 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:39.587713 (+ 3840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.193:40579: BlockingRecv error: recv got EOF from 127.31.59.193:40579 (error 108)
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":232,"threads_started":1}
Aug 10 06:38:39 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:39 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:39.625751 15552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:39.616880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.195:46833)
0810 06:38:39.617405 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:39.617426 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:39.617545 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:39.617809 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:39.617812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:39.618228 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:39.618395 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:39.618405 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:39.619359 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:39.619362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:39.620291 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:39.620304 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:39.620918 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:39.620922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:39.621055 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:39.621560 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:39.621577 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:39.625443 (+ 3866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":278,"threads_started":1}
W20260810 06:38:39.625855 15550 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:39.616949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:46833 (local address 127.31.59.254:40871)
0810 06:38:39.617106 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:39.617111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:39.617489 (+ 378us) server_negotiation.cc:407] Connection header received
0810 06:38:39.617646 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:39.617649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:39.617708 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:39.617776 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:39.618503 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:39.619250 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:39.620408 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:39.620798 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:39.620889 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:39.625634 (+ 4745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.195:46833: BlockingRecv error: recv got EOF from 127.31.59.195:46833 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 10 06:38:39 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:39 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:39.993575 15553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:39.985042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.194:60395)
0810 06:38:39.985544 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:39.985560 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:39.985695 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:39.986040 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:39.986045 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:39.986485 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:39.986653 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:39.986663 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:39.987670 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:39.987674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:39.988660 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:39.988673 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:39.989404 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:39.989410 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:39.989540 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:39.990064 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:39.990080 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:39.993316 (+ 3236us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":279,"threads_started":1}
W20260810 06:38:39.993732 15550 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:39.985118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:60395 (local address 127.31.59.254:40871)
0810 06:38:39.985327 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:39.985333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:39.985649 (+ 316us) server_negotiation.cc:407] Connection header received
0810 06:38:39.985820 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:39.985824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:39.985887 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:39.985966 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:39.986800 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:39.987535 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:39.988800 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:39.989230 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:39.989361 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:39.993502 (+ 4141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.194:60395: BlockingRecv error: recv got EOF from 127.31.59.194:60395 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 06:38:40 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:40 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:40.596925 15555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:40.588608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.193:50777)
0810 06:38:40.589129 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:40.589150 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:40.589268 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:40.589581 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:40.589587 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:40.590025 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:40.590205 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:40.590214 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:40.591273 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:40.591277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:40.592173 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:40.592185 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:40.592893 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:40.592897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:40.593019 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:40.593566 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:40.593582 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:40.596708 (+ 3126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":241,"threads_started":1}
W20260810 06:38:40.597149 15556 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:40.588801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:50777 (local address 127.31.59.254:40871)
0810 06:38:40.589299 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:40.589303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:40.589318 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:38:40.589361 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:40.589364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:40.589422 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:40.589513 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:40.590346 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:40.591162 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:40.592321 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:40.592755 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:40.592856 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:40.596891 (+ 4035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.193:50777: BlockingRecv error: recv got EOF from 127.31.59.193:50777 (error 108)
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":272,"threads_started":1}
Aug 10 06:38:40 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:40 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:40.636969 15558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:40.626923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.195:44941)
0810 06:38:40.627710 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:40.627736 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:40.627878 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:40.628313 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:40.628319 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:40.628856 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:40.629107 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:40.629122 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:40.630548 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:40.630553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:40.631577 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:40.631590 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:40.632294 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:40.632299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:40.632428 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:40.633077 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:40.633097 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:40.636629 (+ 3532us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":538,"threads_started":1}
W20260810 06:38:40.637024 15556 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:40.627025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:44941 (local address 127.31.59.254:40871)
0810 06:38:40.627289 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:40.627296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:40.627825 (+ 529us) server_negotiation.cc:407] Connection header received
0810 06:38:40.628050 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:40.628056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:40.628148 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:40.628261 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:40.629303 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:40.630352 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:40.631739 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:40.632144 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:40.632247 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:40.636808 (+ 4561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.195:44941: BlockingRecv error: recv got EOF from 127.31.59.195:44941 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 10 06:38:40 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:40 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:41.003238 15559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:40.994440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.194:57949)
0810 06:38:40.994899 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:40.994915 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:40.995027 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:40.995379 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:40.995384 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:40.995817 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:40.995993 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:40.996003 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:40.997260 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:40.997266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:40.998335 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:40.998348 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:40.999115 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:40.999120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:40.999240 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:40.999793 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:40.999812 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:41.003013 (+ 3201us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":248,"threads_started":1}
W20260810 06:38:41.003464 15556 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:40.994580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:57949 (local address 127.31.59.254:40871)
0810 06:38:40.994734 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:40.994739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:40.994977 (+ 238us) server_negotiation.cc:407] Connection header received
0810 06:38:40.995175 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:40.995179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:40.995245 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:40.995330 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:40.996132 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:40.997126 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:40.998466 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:40.998974 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:40.999084 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:41.003222 (+ 4138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.194:57949: BlockingRecv error: recv got EOF from 127.31.59.194:57949 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 06:38:41 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:41 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:41.606853 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:41.597929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.193:37981)
0810 06:38:41.598512 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:41.598527 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:41.598649 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:41.598920 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:41.598923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:41.599336 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:41.599521 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:41.599530 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:41.600596 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:41.600601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:41.601513 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:41.601526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:41.602334 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:41.602351 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:41.602481 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:41.603055 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:41.603072 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:41.606501 (+ 3429us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":303,"threads_started":1}
W20260810 06:38:41.606971 15561 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:41.597966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:37981 (local address 127.31.59.254:40871)
0810 06:38:41.598451 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:41.598455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:41.598600 (+ 145us) server_negotiation.cc:407] Connection header received
0810 06:38:41.598735 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:41.598739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:41.598805 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:41.598892 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:41.599668 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:41.600447 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:41.601672 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:41.602150 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:41.602269 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:41.606732 (+ 4463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.193:37981: BlockingRecv error: recv got EOF from 127.31.59.193:37981 (error 108)
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":249,"threads_started":1}
Aug 10 06:38:41 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:41 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:41.646637 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:41.637868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.195:48875)
0810 06:38:41.638397 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:41.638420 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:41.638550 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:41.638830 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:41.638833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:41.639223 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:41.639395 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:41.639404 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:41.640390 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:41.640396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:41.641306 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:41.641320 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:41.642114 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:41.642120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:41.642366 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:41.642992 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:41.643021 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:41.646408 (+ 3387us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":254,"threads_started":1}
W20260810 06:38:41.646759 15561 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:41.637984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:48875 (local address 127.31.59.254:40871)
0810 06:38:41.638193 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:41.638198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:41.638515 (+ 317us) server_negotiation.cc:407] Connection header received
0810 06:38:41.638650 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:41.638653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:41.638720 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:41.638796 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:41.639535 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:41.640256 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:41.641454 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:41.641958 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:41.642062 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:41.646566 (+ 4504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.195:48875: BlockingRecv error: recv got EOF from 127.31.59.195:48875 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 10 06:38:42 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:42 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:42.016075 15563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:42.004108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.194:53547)
0810 06:38:42.004637 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:42.004653 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:42.004764 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:42.005148 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:42.005154 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:42.005596 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:42.005775 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:42.005785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:42.006838 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:42.006842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:42.007768 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:42.007781 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:42.008455 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:42.008459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:42.008603 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:42.012413 (+ 3810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:42.012432 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:42.015830 (+ 3398us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":268,"threads_started":1}
W20260810 06:38:42.016256 15561 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:42.004192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:53547 (local address 127.31.59.254:40871)
0810 06:38:42.004347 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:42.004352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:42.004728 (+ 376us) server_negotiation.cc:407] Connection header received
0810 06:38:42.004923 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:42.004927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:42.004995 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:42.005081 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:42.005885 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:42.006702 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:42.007890 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:42.008322 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:42.008411 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:42.016015 (+ 7604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.194:53547: BlockingRecv error: recv got EOF from 127.31.59.194:53547 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 06:38:42 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:42 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:42.616464 15564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:42.607818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.193:43343)
0810 06:38:42.608280 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:42.608297 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:42.608403 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:42.608853 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:42.608857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:42.609245 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:42.609394 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:42.609402 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:42.610382 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:42.610386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:42.611434 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:42.611447 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:42.612310 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:42.612316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:42.612487 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:42.613019 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:42.613038 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:42.616244 (+ 3206us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":238,"threads_started":1}
W20260810 06:38:42.616664 15566 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:42.607944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:43343 (local address 127.31.59.254:40871)
0810 06:38:42.608582 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:42.608586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:42.608602 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:38:42.608642 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:42.608646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:42.608713 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:42.608797 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:42.609503 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:42.610231 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:42.611594 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:42.612162 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:42.612282 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:42.616430 (+ 4148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.193:43343: BlockingRecv error: recv got EOF from 127.31.59.193:43343 (error 108)
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":416,"threads_started":1}
Aug 10 06:38:42 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:42 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:42.655874 15569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:42.647587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.195:50833)
0810 06:38:42.648132 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:42.648154 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:42.648270 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:42.648529 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:42.648554 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:42.648950 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:42.649122 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:42.649132 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:42.650115 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:42.650119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:42.651104 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:42.651119 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:42.651813 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:42.651817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:42.651950 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:42.652501 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:42.652519 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:42.655603 (+ 3084us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":280,"threads_started":1}
W20260810 06:38:42.655932 15566 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:42.647669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:50833 (local address 127.31.59.254:40871)
0810 06:38:42.647834 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:42.647841 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:42.648237 (+ 396us) server_negotiation.cc:407] Connection header received
0810 06:38:42.648360 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:42.648364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:42.648426 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:42.648500 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:42.649251 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:42.649979 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:42.651214 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:42.651677 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:42.651783 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:42.655765 (+ 3982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.195:50833: BlockingRecv error: recv got EOF from 127.31.59.195:50833 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 10 06:38:43 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:43 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:43.027599 15570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:43.017006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.194:57079)
0810 06:38:43.017541 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:43.017558 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:43.017695 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:43.018022 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:43.018036 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:43.018514 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:43.018679 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:43.018688 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:43.019841 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:43.019849 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:43.021365 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:43.021383 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:43.022315 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:43.022323 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:43.022505 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:43.023189 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:43.023210 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:43.027277 (+ 4067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":307,"threads_started":1}
W20260810 06:38:43.027940 15566 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:43.017169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:57079 (local address 127.31.59.254:40871)
0810 06:38:43.017386 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:43.017392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:43.017640 (+ 248us) server_negotiation.cc:407] Connection header received
0810 06:38:43.017806 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:43.017810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:43.017885 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:43.017968 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:43.018816 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:43.019663 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:43.021505 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:43.022147 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:43.022264 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:43.027519 (+ 5255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.194:57079: BlockingRecv error: recv got EOF from 127.31.59.194:57079 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 10 06:38:43 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:43 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.193@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:43.626157 15573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:43.617580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.193:60287)
0810 06:38:43.618089 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:43.618105 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:43.618216 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:43.618597 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:43.618602 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:43.618996 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:43.619179 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:43.619189 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:43.620214 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:43.620218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:43.621142 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:43.621155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:43.621835 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:43.621842 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:43.622037 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:43.622605 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:43.622623 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:43.625909 (+ 3286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":256,"threads_started":1}
W20260810 06:38:43.626358 15576 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:43.617664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:60287 (local address 127.31.59.254:40871)
0810 06:38:43.618110 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:43.618116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:43.618163 (+ 47us) server_negotiation.cc:407] Connection header received
0810 06:38:43.618366 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:43.618370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:43.618451 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:43.618540 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:43.619341 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:43.620077 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:43.621278 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:43.621697 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:43.621803 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:43.626103 (+ 4300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.193:60287: BlockingRecv error: recv got EOF from 127.31.59.193:60287 (error 108)
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":214,"threads_started":1}
Aug 10 06:38:43 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:43 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.195@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:43.665145 15579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:43.656740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.195:40301)
0810 06:38:43.657232 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:43.657252 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:43.657368 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:43.657647 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:43.657651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:43.658037 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:43.658194 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:43.658204 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:43.659208 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:43.659212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:43.660113 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:43.660125 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:43.660785 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:43.660789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:43.660909 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:43.661454 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:43.661470 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:43.664812 (+ 3342us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":278,"threads_started":1}
W20260810 06:38:43.665247 15576 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:43.656805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:40301 (local address 127.31.59.254:40871)
0810 06:38:43.657078 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:43.657083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:43.657332 (+ 249us) server_negotiation.cc:407] Connection header received
0810 06:38:43.657456 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:43.657460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:43.657527 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:43.657611 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:43.658334 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:43.659089 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:43.660267 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:43.660674 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:43.660774 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:43.665001 (+ 4227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.195:40301: BlockingRecv error: recv got EOF from 127.31.59.195:40301 (error 108)
Metrics: {"server-negotiator.queue_time_us":146}
Aug 10 06:38:44 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:38:44 dist-test-slave-4tdj krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.59.194@KRBTEST.COM for oryx/127.31.59.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:38:44.037796 15580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:44.028903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40871 (local address 127.31.59.194:60729)
0810 06:38:44.029463 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:44.029485 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:44.029708 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:44.030060 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:44.030064 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:44.030592 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:44.030786 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:44.030796 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:44.031810 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:44.031816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:44.032725 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:44.032736 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:44.033389 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:44.033395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:44.033577 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:44.034154 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:44.034173 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:44.037428 (+ 3255us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.59.254:40871: Server oryx/127.31.59.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":293,"threads_started":1}
W20260810 06:38:44.037884 15576 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:44.029004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:60729 (local address 127.31.59.254:40871)
0810 06:38:44.029190 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:44.029196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:44.029600 (+ 404us) server_negotiation.cc:407] Connection header received
0810 06:38:44.029830 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:44.029835 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:44.029916 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:44.030018 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:44.030934 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:44.031674 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:44.032863 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:44.033257 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:44.033361 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:44.037646 (+ 4285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.59.194:60729: BlockingRecv error: recv got EOF from 127.31.59.194:60729 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:38:44.534808 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 14702
I20260810 06:38:44.546811 15584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:44.535836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58848 (local address 127.31.59.193:37725)
0810 06:38:44.536477 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:44.536484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:44.536512 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:38:44.536575 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:44.536582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:44.536801 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:44.536970 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:44.538420 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:44.539470 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:44.540582 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:44.541121 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:44.541238 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:44.543864 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:44.543872 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:44.543885 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:44.543926 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:44.546262 (+ 2336us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.31.59.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 06:38:44.546597 (+ 335us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58848: Request ticket server kudu/127.31.59.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":317,"threads_started":1}
W20260810 06:38:44.546916 15584 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58848: Request ticket server kudu/127.31.59.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 06:38:44.547019 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 14702: Not authorized: Client connection negotiation failed: client connection to 127.31.59.193:37725: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.31.59.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 06:38:44.547230 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 14702
I20260810 06:38:44.573175 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 15000
I20260810 06:38:44.585494 15585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:44.574404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55254 (local address 127.31.59.194:39365)
0810 06:38:44.575069 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:44.575078 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:44.575109 (+ 31us) server_negotiation.cc:407] Connection header received
0810 06:38:44.575189 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:44.575198 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:44.575519 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:44.575760 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:44.576965 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:44.578113 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:44.579411 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:44.579976 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:44.580077 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:44.582377 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:44.582386 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:44.582399 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:44.582462 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:44.584993 (+ 2531us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.31.59.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 06:38:44.585276 (+ 283us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55254: Request ticket server kudu/127.31.59.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":336,"threads_started":1}
W20260810 06:38:44.585599 15585 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55254: Request ticket server kudu/127.31.59.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 06:38:44.585589 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 15000: Not authorized: Client connection negotiation failed: client connection to 127.31.59.194:39365: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.31.59.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 06:38:44.585744 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 15000
I20260810 06:38:44.614634 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 15244
I20260810 06:38:44.627178 15586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:44.615803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36668 (local address 127.31.59.195:39153)
0810 06:38:44.616406 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:44.616413 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:44.616439 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:38:44.616517 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:44.616525 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:44.616784 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:44.616945 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:44.618774 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:44.619781 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:44.620971 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:44.621542 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:44.621652 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:44.624150 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:44.624158 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:44.624171 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:44.624212 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:44.626650 (+ 2438us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.31.59.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 06:38:44.626925 (+ 275us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36668: Request ticket server kudu/127.31.59.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":294,"threads_started":1}
W20260810 06:38:44.627285 15586 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36668: Request ticket server kudu/127.31.59.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 06:38:44.627370 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 15244: Not authorized: Client connection negotiation failed: client connection to 127.31.59.195:39153: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.31.59.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 06:38:44.627527 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 15244
I20260810 06:38:44.653692 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 14580
I20260810 06:38:44.655064 14691 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:44.772605 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 14580
2026-08-10T06:38:44Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21419 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T06:38:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:38:44Z Disabled control of system clock
I20260810 06:38:44.859143 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40787
--webserver_interface=127.31.59.254
--webserver_port=0
--builtin_ntp_servers=127.31.59.212:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:40787
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 06:38:45.182670 15595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:45.182945 15595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:45.183039 15595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:45.189991 15595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:45.190092 15595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:45.190129 15595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:45.190184 15595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:45.190229 15595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:45.199113 15595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:40787
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:45.200353 15595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:45.202085 15595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:45.213925 15602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:45.213956 15607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:45.213995 15603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:45.214197 15595 server_base.cc:1034] running on GCE node
I20260810 06:38:45.215277 15595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:45.216642 15595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:45.217828 15595 hybrid_clock.cc:648] HybridClock initialized: now 1786343925217774 us; error 41 us; skew 500 ppm
I20260810 06:38:45.218189 15595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:38:45.220413 15595 webserver.cc:460] Webserver started at http://127.31.59.254:34339/ using document root <none> and password file <none>
I20260810 06:38:45.221074 15595 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:45.221164 15595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:45.221460 15595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:45.224354 15595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "c60bfae8361a4edcb4730989c317c486"
format_stamp: "Formatted at 2026-08-10 06:38:45 on dist-test-slave-4tdj"
server_key: "2e49c748f1b41819a9d915e8ae85f203"
server_key_iv: "1bbc5a2f9285809734e45cb756d50359"
server_key_version: "encryptionkey@0"
I20260810 06:38:45.225114 15595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "c60bfae8361a4edcb4730989c317c486"
format_stamp: "Formatted at 2026-08-10 06:38:45 on dist-test-slave-4tdj"
server_key: "2e49c748f1b41819a9d915e8ae85f203"
server_key_iv: "1bbc5a2f9285809734e45cb756d50359"
server_key_version: "encryptionkey@0"
I20260810 06:38:45.230026 15595 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 06:38:45.233430 15618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:45.234927 15595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 06:38:45.235110 15595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "c60bfae8361a4edcb4730989c317c486"
format_stamp: "Formatted at 2026-08-10 06:38:45 on dist-test-slave-4tdj"
server_key: "2e49c748f1b41819a9d915e8ae85f203"
server_key_iv: "1bbc5a2f9285809734e45cb756d50359"
server_key_version: "encryptionkey@0"
I20260810 06:38:45.235307 15595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:45.261188 15595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:45.262002 15595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:45.262259 15595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:45.278139 15595 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:40787
I20260810 06:38:45.278159 15696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:40787 every 8 connection(s)
I20260810 06:38:45.279429 15595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:38:45.280447 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 15595
I20260810 06:38:45.280664 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:38:45.281315 31983 external_mini_cluster.cc:1428] Setting key 0463ed62db9e323383f33fc284afd829
I20260810 06:38:45.283335 15697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:45.296520 15705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:45.283398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41650 (local address 127.31.59.254:40787)
0810 06:38:45.284505 (+ 1107us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:45.284520 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:45.284668 (+ 148us) server_negotiation.cc:407] Connection header received
0810 06:38:45.286671 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:45.286697 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:45.287280 (+ 583us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:45.287728 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:45.288237 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:45.290168 (+ 1931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:45.291542 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:45.292513 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:45.292695 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:45.293797 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:45.293807 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:45.293824 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:38:45.294558 (+ 734us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:45.294644 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:38:45.294672 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:45.294801 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:45.294960 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 06:38:45.295291 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":349,"threads_started":1}
I20260810 06:38:45.297644 15697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486: Bootstrap starting.
I20260810 06:38:45.300625 15697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:45.301576 15697 log.cc:824] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:45.304250 15697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486: No bootstrap required, opened a new log
I20260810 06:38:45.311262 15697 raft_consensus.cc:348] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c60bfae8361a4edcb4730989c317c486" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40787 } }
I20260810 06:38:45.311601 15697 raft_consensus.cc:374] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:45.311697 15697 raft_consensus.cc:729] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60bfae8361a4edcb4730989c317c486, State: Initialized, Role: FOLLOWER
I20260810 06:38:45.312270 15697 consensus_queue.cc:260] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c60bfae8361a4edcb4730989c317c486" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40787 } }
I20260810 06:38:45.312476 15697 raft_consensus.cc:388] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:38:45.312613 15697 raft_consensus.cc:482] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:38:45.312759 15697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:45.315065 15697 raft_consensus.cc:504] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c60bfae8361a4edcb4730989c317c486" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40787 } }
I20260810 06:38:45.315541 15697 leader_election.cc:302] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c60bfae8361a4edcb4730989c317c486; no voters:
I20260810 06:38:45.316098 15697 leader_election.cc:288] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:38:45.316252 15709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:45.316898 15709 raft_consensus.cc:686] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486 [term 1 LEADER]: Becoming Leader. State: Replica: c60bfae8361a4edcb4730989c317c486, State: Running, Role: LEADER
I20260810 06:38:45.317423 15709 consensus_queue.cc:237] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c60bfae8361a4edcb4730989c317c486" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40787 } }
I20260810 06:38:45.318015 15697 sys_catalog.cc:564] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:45.320291 15711 sys_catalog.cc:455] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c60bfae8361a4edcb4730989c317c486. Latest consensus state: current_term: 1 leader_uuid: "c60bfae8361a4edcb4730989c317c486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c60bfae8361a4edcb4730989c317c486" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40787 } } }
I20260810 06:38:45.320753 15711 sys_catalog.cc:458] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:45.321079 15710 sys_catalog.cc:455] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c60bfae8361a4edcb4730989c317c486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c60bfae8361a4edcb4730989c317c486" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40787 } } }
I20260810 06:38:45.322525 15710 sys_catalog.cc:458] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:45.327339 15722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:38:45.332846 15722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:38:45.341569 15722 catalog_manager.cc:1347] Generated new cluster ID: 40eae3b8dc9c4c91bd91cd28a740e8c8
I20260810 06:38:45.341733 15722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:38:45.364169 15722 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:38:45.366362 15722 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:38:45.380292 15722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c60bfae8361a4edcb4730989c317c486: Generated new TSK 0
I20260810 06:38:45.381074 15722 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:38:45.392015 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40787
--builtin_ntp_servers=127.31.59.212:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 06:38:45.738659 15737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:45.738909 15737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:45.739024 15737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:45.746038 15737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:45.746124 15737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:45.746259 15737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:38:45.754837 15737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:45.755915 15737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:45.757678 15737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:45.769639 15748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:45.769716 15750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:45.769670 15754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:45.770828 15737 server_base.cc:1034] running on GCE node
I20260810 06:38:45.771364 15737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:45.772212 15737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:45.773406 15737 hybrid_clock.cc:648] HybridClock initialized: now 1786343925773359 us; error 36 us; skew 500 ppm
I20260810 06:38:45.773695 15737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:38:45.775759 15737 webserver.cc:460] Webserver started at http://127.31.59.193:44335/ using document root <none> and password file <none>
I20260810 06:38:45.776376 15737 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:45.776475 15737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:45.776791 15737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:45.779492 15737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "d24d8849f530443a88fa2cbde6171b2e"
format_stamp: "Formatted at 2026-08-10 06:38:45 on dist-test-slave-4tdj"
server_key: "101defe9ad6d80bf67580b6d4a33e1f9"
server_key_iv: "8dfe4bea57ed7421694ecc4c91cc6618"
server_key_version: "encryptionkey@0"
I20260810 06:38:45.780231 15737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "d24d8849f530443a88fa2cbde6171b2e"
format_stamp: "Formatted at 2026-08-10 06:38:45 on dist-test-slave-4tdj"
server_key: "101defe9ad6d80bf67580b6d4a33e1f9"
server_key_iv: "8dfe4bea57ed7421694ecc4c91cc6618"
server_key_version: "encryptionkey@0"
I20260810 06:38:45.784919 15737 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:38:45.788126 15765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:45.789304 15737 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:38:45.789449 15737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "d24d8849f530443a88fa2cbde6171b2e"
format_stamp: "Formatted at 2026-08-10 06:38:45 on dist-test-slave-4tdj"
server_key: "101defe9ad6d80bf67580b6d4a33e1f9"
server_key_iv: "8dfe4bea57ed7421694ecc4c91cc6618"
server_key_version: "encryptionkey@0"
I20260810 06:38:45.789656 15737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:45.805307 15737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:45.806017 15737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:45.806272 15737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:45.807153 15737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:45.808450 15737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:45.808552 15737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:45.808651 15737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:45.808722 15737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:45.838702 15737 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:34969
I20260810 06:38:45.838764 15962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:34969 every 8 connection(s)
I20260810 06:38:45.839979 15737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:38:45.840973 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 15737
I20260810 06:38:45.841198 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:38:45.841960 31983 external_mini_cluster.cc:1428] Setting key 3a37c5c38747aa954d7221476019cbd3
I20260810 06:38:45.844686 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40787
--builtin_ntp_servers=127.31.59.212:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 06:38:45.852497 15967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:45.842815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:44405 (local address 127.31.59.254:40787)
0810 06:38:45.843282 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:45.843287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:45.844613 (+ 1326us) server_negotiation.cc:407] Connection header received
0810 06:38:45.846384 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:45.846389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:45.846469 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:45.846566 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:45.847636 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:45.848356 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:45.849873 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:45.850283 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:45.850423 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:45.851701 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:45.851706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:45.851710 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:38:45.851727 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:45.851766 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:38:45.851778 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:45.851850 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:45.852059 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 06:38:45.852155 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":249,"threads_started":1}
I20260810 06:38:45.853436 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:45.843445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40787 (local address 127.31.59.193:44405)
0810 06:38:45.844452 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:45.844516 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:45.846116 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:45.846815 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:45.846824 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:45.846909 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:45.847466 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:45.847494 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:45.848513 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:45.848517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:45.849733 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:45.849745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:45.850528 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:45.850536 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:45.850757 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:45.851392 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:38:45.851395 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:45.851435 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:38:45.851518 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:45.851864 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:45.851875 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:45.851883 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:38:45.852036 (+ 153us) client_negotiation.cc:240] Negotiation successful
0810 06:38:45.852410 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":304,"threads_started":1}
I20260810 06:38:45.855048 15963 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40787
I20260810 06:38:45.855312 15963 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:45.856076 15963 heartbeater.cc:511] Master 127.31.59.254:40787 requested a full tablet report, sending...
I20260810 06:38:45.858114 15639 ts_manager.cc:194] Registered new tserver with Master: d24d8849f530443a88fa2cbde6171b2e (127.31.59.193:34969)
I20260810 06:38:45.859607 15639 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.193:44405
W20260810 06:38:46.179210 15971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:46.179503 15971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:46.179643 15971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:46.186924 15971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:46.187037 15971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:46.187188 15971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:38:46.196151 15971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:46.197361 15971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:46.199215 15971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:46.211900 15981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:46.211962 15979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:46.211941 15977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:46.212615 15971 server_base.cc:1034] running on GCE node
I20260810 06:38:46.213228 15971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:46.214171 15971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:46.215376 15971 hybrid_clock.cc:648] HybridClock initialized: now 1786343926215314 us; error 48 us; skew 500 ppm
I20260810 06:38:46.215694 15971 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:38:46.217895 15971 webserver.cc:460] Webserver started at http://127.31.59.194:46063/ using document root <none> and password file <none>
I20260810 06:38:46.218648 15971 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:46.218767 15971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:46.219113 15971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:46.222036 15971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "774b62d1c79148b1a098b10e6a5213c3"
format_stamp: "Formatted at 2026-08-10 06:38:46 on dist-test-slave-4tdj"
server_key: "65e1c825ed8cc362cc0ac19adc08ce2f"
server_key_iv: "2a0a6537506b7ce85886ff7d124e7942"
server_key_version: "encryptionkey@0"
I20260810 06:38:46.222896 15971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "774b62d1c79148b1a098b10e6a5213c3"
format_stamp: "Formatted at 2026-08-10 06:38:46 on dist-test-slave-4tdj"
server_key: "65e1c825ed8cc362cc0ac19adc08ce2f"
server_key_iv: "2a0a6537506b7ce85886ff7d124e7942"
server_key_version: "encryptionkey@0"
I20260810 06:38:46.228191 15971 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:38:46.231459 15990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:46.232755 15971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:38:46.232916 15971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "774b62d1c79148b1a098b10e6a5213c3"
format_stamp: "Formatted at 2026-08-10 06:38:46 on dist-test-slave-4tdj"
server_key: "65e1c825ed8cc362cc0ac19adc08ce2f"
server_key_iv: "2a0a6537506b7ce85886ff7d124e7942"
server_key_version: "encryptionkey@0"
I20260810 06:38:46.233136 15971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:46.246932 15971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:46.247697 15971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:46.247972 15971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:46.248865 15971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:46.250255 15971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:46.250375 15971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:46.250479 15971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:46.250554 15971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:46.281715 15971 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:42771
I20260810 06:38:46.281775 16174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:42771 every 8 connection(s)
I20260810 06:38:46.283252 15971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:38:46.289956 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 15971
I20260810 06:38:46.290141 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:38:46.290681 31983 external_mini_cluster.cc:1428] Setting key 4fcbe20fc7a6e948e620ebb0f622e405
I20260810 06:38:46.293804 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40787
--builtin_ntp_servers=127.31.59.212:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 06:38:46.297407 15967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:46.285931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:49109 (local address 127.31.59.254:40787)
0810 06:38:46.286100 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:46.286104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:46.287781 (+ 1677us) server_negotiation.cc:407] Connection header received
0810 06:38:46.289624 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:46.289631 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:46.289709 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:46.289806 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:46.291073 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:46.292051 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:46.294337 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:46.294983 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:46.295126 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:46.296588 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:46.296595 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:46.296599 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:38:46.296621 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:46.296668 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:38:46.296685 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:46.296787 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:46.296996 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 06:38:46.297093 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:38:46.298204 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:46.286587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40787 (local address 127.31.59.194:49109)
0810 06:38:46.287610 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:46.287663 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:46.289323 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:46.290071 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:46.290083 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:46.290191 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:46.290882 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:46.290902 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:46.292208 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:46.292214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:46.294154 (+ 1940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:46.294170 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:46.295197 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:46.295204 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:46.295486 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:46.296233 (+ 747us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:38:46.296237 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:46.296284 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:38:46.296372 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:46.296801 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:46.296810 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:46.296816 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 06:38:46.296958 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 06:38:46.297224 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":335,"threads_started":1}
I20260810 06:38:46.299481 16175 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40787
I20260810 06:38:46.299793 16175 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:46.300711 16175 heartbeater.cc:511] Master 127.31.59.254:40787 requested a full tablet report, sending...
I20260810 06:38:46.302217 15639 ts_manager.cc:194] Registered new tserver with Master: 774b62d1c79148b1a098b10e6a5213c3 (127.31.59.194:42771)
I20260810 06:38:46.302991 15639 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.194:49109
W20260810 06:38:46.613020 16180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:46.613276 16180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:46.613395 16180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:46.620385 16180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:46.620476 16180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:46.620626 16180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:38:46.629236 16180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:46.630375 16180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:46.632014 16180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:46.644291 16186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:46.644356 16190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:46.644389 16188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:46.645105 16180 server_base.cc:1034] running on GCE node
I20260810 06:38:46.646281 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:46.647389 16180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:46.648694 16180 hybrid_clock.cc:648] HybridClock initialized: now 1786343926648631 us; error 47 us; skew 500 ppm
I20260810 06:38:46.649052 16180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:38:46.651108 16180 webserver.cc:460] Webserver started at http://127.31.59.195:46735/ using document root <none> and password file <none>
I20260810 06:38:46.651743 16180 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:46.651839 16180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:46.652138 16180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:46.654850 16180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "eca31c25f4744f868ea1db00dfd5ae37"
format_stamp: "Formatted at 2026-08-10 06:38:46 on dist-test-slave-4tdj"
server_key: "54fb4cfe41a85abdf8a4e396de8e1c2b"
server_key_iv: "cea5c81d794205d09dd7f36028a8b17d"
server_key_version: "encryptionkey@0"
I20260810 06:38:46.655586 16180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "eca31c25f4744f868ea1db00dfd5ae37"
format_stamp: "Formatted at 2026-08-10 06:38:46 on dist-test-slave-4tdj"
server_key: "54fb4cfe41a85abdf8a4e396de8e1c2b"
server_key_iv: "cea5c81d794205d09dd7f36028a8b17d"
server_key_version: "encryptionkey@0"
I20260810 06:38:46.660588 16180 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 06:38:46.663745 16199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:46.664959 16180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:38:46.665110 16180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "eca31c25f4744f868ea1db00dfd5ae37"
format_stamp: "Formatted at 2026-08-10 06:38:46 on dist-test-slave-4tdj"
server_key: "54fb4cfe41a85abdf8a4e396de8e1c2b"
server_key_iv: "cea5c81d794205d09dd7f36028a8b17d"
server_key_version: "encryptionkey@0"
I20260810 06:38:46.665313 16180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:46.685256 16180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:46.685998 16180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:46.686270 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:46.687167 16180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:46.688520 16180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:46.688613 16180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:46.688714 16180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:46.688784 16180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:46.718559 16180 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:45755
I20260810 06:38:46.718618 16385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:45755 every 8 connection(s)
I20260810 06:38:46.719746 16180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:38:46.728914 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 16180
I20260810 06:38:46.729111 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:38:46.729691 31983 external_mini_cluster.cc:1428] Setting key 7ed166d46b827097d28ec9bcf4a43601
I20260810 06:38:46.732220 15967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:46.722244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:35537 (local address 127.31.59.254:40787)
0810 06:38:46.722469 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:46.722473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:46.723822 (+ 1349us) server_negotiation.cc:407] Connection header received
0810 06:38:46.725570 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:46.725574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:46.725642 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:46.725738 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:46.726942 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:46.727696 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:46.729215 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:46.729903 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:46.730056 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:46.731351 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:46.731356 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:46.731359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:38:46.731378 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:46.731412 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:38:46.731426 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:46.731553 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:46.731778 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 06:38:46.731877 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260810 06:38:46.733179 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:46.722781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40787 (local address 127.31.59.195:35537)
0810 06:38:46.723677 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:46.723713 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:46.725280 (+ 1567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:46.725961 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:46.725970 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:46.726068 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:46.726782 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:46.726802 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:46.727833 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:46.727837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:46.729039 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:46.729051 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:46.730107 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:46.730112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:46.730334 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:46.731045 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:38:46.731048 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:46.731084 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:38:46.731169 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:46.731563 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:46.731573 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:46.731582 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 06:38:46.731724 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 06:38:46.732091 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":263,"threads_started":1}
I20260810 06:38:46.734445 16386 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40787
I20260810 06:38:46.734721 16386 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:46.735420 16386 heartbeater.cc:511] Master 127.31.59.254:40787 requested a full tablet report, sending...
I20260810 06:38:46.736763 15639 ts_manager.cc:194] Registered new tserver with Master: eca31c25f4744f868ea1db00dfd5ae37 (127.31.59.195:45755)
I20260810 06:38:46.737615 15639 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.195:35537
I20260810 06:38:46.745184 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 06:38:46.751037 15967 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:46.749536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41662 (local address 127.31.59.254:40787)
0810 06:38:46.749718 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:46.749725 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:46.750118 (+ 393us) server_negotiation.cc:407] Connection header received
0810 06:38:46.750240 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:46.750244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:46.750358 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:46.750432 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:46.750825 (+ 393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41662: BlockingRecv error: recv got EOF from 127.0.0.1:41662 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:38:46.752044 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 15737
I20260810 06:38:46.758922 16397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:46.752788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35378 (local address 127.31.59.193:34969)
0810 06:38:46.753225 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:46.753231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:46.753255 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:38:46.753305 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:46.753310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:46.753481 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:46.753602 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:46.753914 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:46.754901 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:46.756110 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:46.756631 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:46.756723 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:46.757594 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:46.757602 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:46.757613 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:38:46.758135 (+ 522us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:46.758179 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:38:46.758197 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:46.758286 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:46.758442 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 06:38:46.758579 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":210,"threads_started":1}
I20260810 06:38:46.760018 15953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:46.839064 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 15737
I20260810 06:38:46.863897 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 15971
I20260810 06:38:46.871510 16401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:46.864696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44956 (local address 127.31.59.194:42771)
0810 06:38:46.865190 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:46.865198 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:46.865238 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:38:46.865298 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:46.865305 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:46.865517 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:46.865645 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:46.866097 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:46.867159 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:46.868463 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:46.869068 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:46.869174 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:46.870197 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:46.870205 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:46.870218 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:38:46.870817 (+ 599us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:46.870864 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:38:46.870884 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:46.870977 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:46.871105 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 06:38:46.871216 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":234,"threads_started":1}
I20260810 06:38:46.872718 16166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:46.952126 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 15971
I20260810 06:38:46.976554 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 16180
I20260810 06:38:46.984498 16405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:46.977730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42054 (local address 127.31.59.195:45755)
0810 06:38:46.978362 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:46.978369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:46.978401 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:38:46.978464 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:46.978471 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:46.978688 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:46.978812 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:46.979235 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:46.980296 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:46.981494 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:46.982050 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:46.982145 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:46.983219 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:46.983228 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:46.983241 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:38:46.983808 (+ 567us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:46.983856 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:38:46.983876 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:46.983961 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:46.984082 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 06:38:46.984188 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":302,"threads_started":1}
I20260810 06:38:46.985710 16379 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:47.067096 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 16180
I20260810 06:38:47.084882 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 15595
I20260810 06:38:47.086107 15689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:47.186703 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 15595
2026-08-10T06:38:47Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2407 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T06:38:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:38:47Z Disabled control of system clock
I20260810 06:38:47.268774 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:34975
--webserver_interface=127.31.59.254
--webserver_port=0
--builtin_ntp_servers=127.31.59.212:35111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:34975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 06:38:47.592155 16413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:47.592427 16413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:47.592548 16413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:47.599576 16413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:47.599674 16413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:47.599738 16413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:47.599817 16413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:47.599879 16413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:47.608850 16413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:35111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:34975
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:34975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:47.610037 16413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:47.611831 16413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:47.623114 16421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:47.623194 16419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:47.623241 16425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:47.624312 16413 server_base.cc:1034] running on GCE node
I20260810 06:38:47.625052 16413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:47.626397 16413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:47.627631 16413 hybrid_clock.cc:648] HybridClock initialized: now 1786343927627578 us; error 45 us; skew 500 ppm
I20260810 06:38:47.627965 16413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:38:47.633363 16413 webserver.cc:460] Webserver started at http://127.31.59.254:42781/ using document root <none> and password file <none>
I20260810 06:38:47.634100 16413 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:47.634205 16413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:47.634579 16413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:47.637372 16413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "e0c26567fa654e5096e1e9a703fb860d"
format_stamp: "Formatted at 2026-08-10 06:38:47 on dist-test-slave-4tdj"
server_key: "2b1e566032a12077a4210670de61bf6a"
server_key_iv: "c3a34485c8d8b2fa3468dfa048555d9c"
server_key_version: "encryptionkey@0"
I20260810 06:38:47.638154 16413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "e0c26567fa654e5096e1e9a703fb860d"
format_stamp: "Formatted at 2026-08-10 06:38:47 on dist-test-slave-4tdj"
server_key: "2b1e566032a12077a4210670de61bf6a"
server_key_iv: "c3a34485c8d8b2fa3468dfa048555d9c"
server_key_version: "encryptionkey@0"
I20260810 06:38:47.643118 16413 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:38:47.646481 16436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:47.647796 16413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:38:47.647959 16413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "e0c26567fa654e5096e1e9a703fb860d"
format_stamp: "Formatted at 2026-08-10 06:38:47 on dist-test-slave-4tdj"
server_key: "2b1e566032a12077a4210670de61bf6a"
server_key_iv: "c3a34485c8d8b2fa3468dfa048555d9c"
server_key_version: "encryptionkey@0"
I20260810 06:38:47.648156 16413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:47.654887 16413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:47.655639 16413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:47.655916 16413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:47.672870 16413 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:34975
I20260810 06:38:47.672892 16520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:34975 every 8 connection(s)
I20260810 06:38:47.674146 16413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:38:47.677788 16523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:47.679013 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 16413
I20260810 06:38:47.679219 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:38:47.680027 31983 external_mini_cluster.cc:1428] Setting key 01347c4a188b0a5d8e0b2c5af44b9540
I20260810 06:38:47.689857 16527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:47.682352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45418 (local address 127.31.59.254:34975)
0810 06:38:47.683562 (+ 1210us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:47.683580 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:47.683641 (+ 61us) server_negotiation.cc:407] Connection header received
0810 06:38:47.685965 (+ 2324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:47.685991 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:47.686516 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:47.686935 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:47.687939 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:47.687947 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:47.687966 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:38:47.687982 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:47.688057 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:38:47.688092 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:47.688311 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:47.688433 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 06:38:47.688710 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":408,"threads_started":1}
I20260810 06:38:47.691383 16523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d: Bootstrap starting.
I20260810 06:38:47.694489 16523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:47.695403 16523 log.cc:824] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:47.697901 16523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d: No bootstrap required, opened a new log
I20260810 06:38:47.704728 16523 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c26567fa654e5096e1e9a703fb860d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34975 } }
I20260810 06:38:47.705125 16523 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:47.705240 16523 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0c26567fa654e5096e1e9a703fb860d, State: Initialized, Role: FOLLOWER
I20260810 06:38:47.705771 16523 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c26567fa654e5096e1e9a703fb860d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34975 } }
I20260810 06:38:47.705979 16523 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:38:47.706108 16523 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:38:47.706264 16523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:47.708284 16523 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c26567fa654e5096e1e9a703fb860d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34975 } }
I20260810 06:38:47.708758 16523 leader_election.cc:302] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0c26567fa654e5096e1e9a703fb860d; no voters:
I20260810 06:38:47.709509 16523 leader_election.cc:288] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:38:47.709618 16532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:47.710343 16532 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d [term 1 LEADER]: Becoming Leader. State: Replica: e0c26567fa654e5096e1e9a703fb860d, State: Running, Role: LEADER
I20260810 06:38:47.710850 16532 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c26567fa654e5096e1e9a703fb860d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34975 } }
I20260810 06:38:47.711407 16523 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:47.713601 16533 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0c26567fa654e5096e1e9a703fb860d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c26567fa654e5096e1e9a703fb860d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34975 } } }
I20260810 06:38:47.713625 16535 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0c26567fa654e5096e1e9a703fb860d. Latest consensus state: current_term: 1 leader_uuid: "e0c26567fa654e5096e1e9a703fb860d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c26567fa654e5096e1e9a703fb860d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34975 } } }
I20260810 06:38:47.714025 16533 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:47.714030 16535 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:47.716985 16547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:38:47.722100 16547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:38:47.731130 16547 catalog_manager.cc:1347] Generated new cluster ID: 11cfd7077aa14f5a9b18d9521afddc8b
I20260810 06:38:47.731238 16547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:38:47.743007 16547 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:38:47.743383 16547 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:38:47.748761 16547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0c26567fa654e5096e1e9a703fb860d: Generated new TSK 0
I20260810 06:38:47.749461 16547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:38:47.757545 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:34975
--builtin_ntp_servers=127.31.59.212:35111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 06:38:48.089421 16566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:48.089727 16566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:48.089826 16566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:48.096956 16566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:48.097054 16566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:48.097170 16566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:38:48.105995 16566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:35111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:34975
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:48.107237 16566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:48.108955 16566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:48.121730 16575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:48.121707 16578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:48.121707 16576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:48.122604 16566 server_base.cc:1034] running on GCE node
I20260810 06:38:48.123163 16566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:48.124059 16566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:48.125284 16566 hybrid_clock.cc:648] HybridClock initialized: now 1786343928125213 us; error 59 us; skew 500 ppm
I20260810 06:38:48.125622 16566 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:38:48.127777 16566 webserver.cc:460] Webserver started at http://127.31.59.193:35095/ using document root <none> and password file <none>
I20260810 06:38:48.128422 16566 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:48.128535 16566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:48.128845 16566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:48.133810 16566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "d942547c43484bc28fe5db43e4e8023f"
format_stamp: "Formatted at 2026-08-10 06:38:48 on dist-test-slave-4tdj"
server_key: "bcf56af6da80e8fd0423c412f03fc037"
server_key_iv: "7120d8596fb6587b984dad48068f23aa"
server_key_version: "encryptionkey@0"
I20260810 06:38:48.134763 16566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "d942547c43484bc28fe5db43e4e8023f"
format_stamp: "Formatted at 2026-08-10 06:38:48 on dist-test-slave-4tdj"
server_key: "bcf56af6da80e8fd0423c412f03fc037"
server_key_iv: "7120d8596fb6587b984dad48068f23aa"
server_key_version: "encryptionkey@0"
I20260810 06:38:48.139719 16566 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 06:38:48.143036 16585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:48.144281 16566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260810 06:38:48.144441 16566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "d942547c43484bc28fe5db43e4e8023f"
format_stamp: "Formatted at 2026-08-10 06:38:48 on dist-test-slave-4tdj"
server_key: "bcf56af6da80e8fd0423c412f03fc037"
server_key_iv: "7120d8596fb6587b984dad48068f23aa"
server_key_version: "encryptionkey@0"
I20260810 06:38:48.144666 16566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:48.150122 16566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:48.150913 16566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:48.151163 16566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:48.152181 16566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:48.153623 16566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:48.153726 16566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:48.153833 16566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:48.153906 16566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:48.185189 16566 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:45519
I20260810 06:38:48.185259 16766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:45519 every 8 connection(s)
I20260810 06:38:48.186584 16566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:38:48.193082 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 16566
I20260810 06:38:48.193264 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:38:48.194061 31983 external_mini_cluster.cc:1428] Setting key 96df40dcf0aac2d72e09ee38da15ea1d
I20260810 06:38:48.194469 16527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:48.189161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:37001 (local address 127.31.59.254:34975)
0810 06:38:48.189339 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:48.189344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:48.190661 (+ 1317us) server_negotiation.cc:407] Connection header received
0810 06:38:48.192237 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:48.192242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:48.192298 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:48.192384 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:48.193765 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:48.193769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:48.193772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:38:48.193788 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:48.193820 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:38:48.193831 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:48.193917 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:48.194159 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 06:38:48.194225 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:38:48.195523 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:48.189596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:34975 (local address 127.31.59.193:37001)
0810 06:38:48.190512 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:48.190551 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:48.191988 (+ 1437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:48.192571 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:48.192583 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:48.192700 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:48.193407 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:38:48.193411 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:48.193463 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:38:48.193575 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:48.193927 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:48.193937 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:48.193945 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:38:48.194235 (+ 290us) client_negotiation.cc:240] Negotiation successful
0810 06:38:48.194516 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":258,"threads_started":1}
I20260810 06:38:48.197110 16767 heartbeater.cc:347] Connected to a master server at 127.31.59.254:34975
I20260810 06:38:48.197149 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:34975
--builtin_ntp_servers=127.31.59.212:35111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 06:38:48.197417 16767 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:48.198074 16767 heartbeater.cc:511] Master 127.31.59.254:34975 requested a full tablet report, sending...
I20260810 06:38:48.200085 16469 ts_manager.cc:194] Registered new tserver with Master: d942547c43484bc28fe5db43e4e8023f (127.31.59.193:45519)
W20260810 06:38:48.518350 16770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:48.518610 16770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:48.518707 16770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:48.525816 16770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:48.525928 16770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:48.526036 16770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:38:48.535177 16770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:35111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:34975
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:48.536408 16770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:48.538170 16770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:48.550876 16780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:48.550976 16782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:48.550999 16784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:48.552096 16770 server_base.cc:1034] running on GCE node
I20260810 06:38:48.552794 16770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:48.553755 16770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:48.554986 16770 hybrid_clock.cc:648] HybridClock initialized: now 1786343928554911 us; error 48 us; skew 500 ppm
I20260810 06:38:48.555311 16770 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:38:48.557502 16770 webserver.cc:460] Webserver started at http://127.31.59.194:33297/ using document root <none> and password file <none>
I20260810 06:38:48.558209 16770 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:48.558344 16770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:48.558717 16770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:48.561648 16770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "f9669d0a1cdc47c58dadfd6701ccd2c9"
format_stamp: "Formatted at 2026-08-10 06:38:48 on dist-test-slave-4tdj"
server_key: "f38399e55dd15c74a02558bf681b6414"
server_key_iv: "0e896ef7ee0ac958aa7f3801d88ca75e"
server_key_version: "encryptionkey@0"
I20260810 06:38:48.562549 16770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "f9669d0a1cdc47c58dadfd6701ccd2c9"
format_stamp: "Formatted at 2026-08-10 06:38:48 on dist-test-slave-4tdj"
server_key: "f38399e55dd15c74a02558bf681b6414"
server_key_iv: "0e896ef7ee0ac958aa7f3801d88ca75e"
server_key_version: "encryptionkey@0"
I20260810 06:38:48.567700 16770 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:38:48.571105 16792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:48.572381 16770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260810 06:38:48.572546 16770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "f9669d0a1cdc47c58dadfd6701ccd2c9"
format_stamp: "Formatted at 2026-08-10 06:38:48 on dist-test-slave-4tdj"
server_key: "f38399e55dd15c74a02558bf681b6414"
server_key_iv: "0e896ef7ee0ac958aa7f3801d88ca75e"
server_key_version: "encryptionkey@0"
I20260810 06:38:48.572757 16770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:48.578440 16770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:48.579267 16770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:48.579545 16770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:48.580500 16770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:48.581923 16770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:48.582034 16770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:48.582157 16770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:48.582237 16770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:48.615914 16770 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:40841
I20260810 06:38:48.615983 16978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:40841 every 8 connection(s)
I20260810 06:38:48.617337 16770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:38:48.623088 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 16770
I20260810 06:38:48.623258 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:38:48.623735 31983 external_mini_cluster.cc:1428] Setting key d9a9b3cf77fb765e8a0f729542314e3e
I20260810 06:38:48.625417 16527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:48.619826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:49735 (local address 127.31.59.254:34975)
0810 06:38:48.620031 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:48.620035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:48.621401 (+ 1366us) server_negotiation.cc:407] Connection header received
0810 06:38:48.623137 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:48.623142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:48.623222 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:48.623330 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:48.624629 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:48.624635 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:48.624639 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:38:48.624654 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:48.624687 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:38:48.624699 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:48.624787 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:48.625087 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 06:38:48.625142 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 06:38:48.626376 16983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:48.620322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:34975 (local address 127.31.59.194:49735)
0810 06:38:48.621248 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:48.621291 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:48.622865 (+ 1574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:48.623580 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:48.623588 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:48.623680 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:48.624341 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:38:48.624343 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:48.624382 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:38:48.624469 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:48.624800 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:48.624810 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:48.624817 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 06:38:48.625102 (+ 285us) client_negotiation.cc:240] Negotiation successful
0810 06:38:48.625393 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":271,"threads_started":1}
I20260810 06:38:48.626709 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:34975
--builtin_ntp_servers=127.31.59.212:35111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 06:38:48.627868 16979 heartbeater.cc:347] Connected to a master server at 127.31.59.254:34975
I20260810 06:38:48.628152 16979 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:48.628732 16979 heartbeater.cc:511] Master 127.31.59.254:34975 requested a full tablet report, sending...
I20260810 06:38:48.630223 16469 ts_manager.cc:194] Registered new tserver with Master: f9669d0a1cdc47c58dadfd6701ccd2c9 (127.31.59.194:40841)
W20260810 06:38:48.954319 16984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:48.954622 16984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:48.954771 16984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:48.962509 16984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:48.962626 16984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:48.962783 16984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:38:48.971953 16984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:35111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:34975
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:48.973273 16984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:48.975176 16984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:48.988632 16992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:48.988533 16998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:48.988581 16994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:48.989440 16984 server_base.cc:1034] running on GCE node
I20260810 06:38:48.989998 16984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:48.990938 16984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:48.992172 16984 hybrid_clock.cc:648] HybridClock initialized: now 1786343928992122 us; error 33 us; skew 500 ppm
I20260810 06:38:48.992470 16984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:38:48.994669 16984 webserver.cc:460] Webserver started at http://127.31.59.195:40503/ using document root <none> and password file <none>
I20260810 06:38:48.995321 16984 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:48.995422 16984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:48.995743 16984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:48.998584 16984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "6641a866ef054a0690f332a8597ec776"
format_stamp: "Formatted at 2026-08-10 06:38:48 on dist-test-slave-4tdj"
server_key: "b762cee39881e92324f8d03a2005907b"
server_key_iv: "9e0a9c6d6c69d19428e29ad2eebd1720"
server_key_version: "encryptionkey@0"
I20260810 06:38:48.999349 16984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "6641a866ef054a0690f332a8597ec776"
format_stamp: "Formatted at 2026-08-10 06:38:48 on dist-test-slave-4tdj"
server_key: "b762cee39881e92324f8d03a2005907b"
server_key_iv: "9e0a9c6d6c69d19428e29ad2eebd1720"
server_key_version: "encryptionkey@0"
I20260810 06:38:49.004261 16984 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:38:49.007628 17010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:49.008873 16984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:38:49.009038 16984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "6641a866ef054a0690f332a8597ec776"
format_stamp: "Formatted at 2026-08-10 06:38:48 on dist-test-slave-4tdj"
server_key: "b762cee39881e92324f8d03a2005907b"
server_key_iv: "9e0a9c6d6c69d19428e29ad2eebd1720"
server_key_version: "encryptionkey@0"
I20260810 06:38:49.009238 16984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:49.014879 16984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:49.015630 16984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:49.015874 16984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:49.016808 16984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:49.018164 16984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:49.018251 16984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:49.018380 16984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:49.018447 16984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:49.066131 16984 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:37761
I20260810 06:38:49.066179 17192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:37761 every 8 connection(s)
I20260810 06:38:49.067451 16984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:38:49.075325 16527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:49.069891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:33393 (local address 127.31.59.254:34975)
0810 06:38:49.070094 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:49.070098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:49.071530 (+ 1432us) server_negotiation.cc:407] Connection header received
0810 06:38:49.073167 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:49.073173 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:49.073239 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:49.073326 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:49.074628 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:49.074632 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:49.074636 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:38:49.074652 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:49.074684 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:38:49.074695 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:49.074807 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:49.074999 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 06:38:49.075068 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:38:49.075781 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 16984
I20260810 06:38:49.075939 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:38:49.076325 17199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:49.070437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:34975 (local address 127.31.59.195:33393)
0810 06:38:49.071337 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:49.071374 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:49.072892 (+ 1518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:49.073566 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:49.073574 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:49.073662 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:49.074258 (+ 596us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:38:49.074261 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:49.074343 (+ 82us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:38:49.074434 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:49.074790 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:49.074800 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:49.074807 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 06:38:49.075070 (+ 263us) client_negotiation.cc:240] Negotiation successful
0810 06:38:49.075310 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":254,"threads_started":1}
I20260810 06:38:49.076431 31983 external_mini_cluster.cc:1428] Setting key 9d48e4c9b2abc3090ed2fa100a2fba51
I20260810 06:38:49.077832 17196 heartbeater.cc:347] Connected to a master server at 127.31.59.254:34975
I20260810 06:38:49.078135 17196 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:49.078902 17196 heartbeater.cc:511] Master 127.31.59.254:34975 requested a full tablet report, sending...
I20260810 06:38:49.080255 16469 ts_manager.cc:194] Registered new tserver with Master: 6641a866ef054a0690f332a8597ec776 (127.31.59.195:37761)
I20260810 06:38:49.091255 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 06:38:49.096957 16527 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:49.095519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57148 (local address 127.31.59.254:34975)
0810 06:38:49.095676 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:49.095680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:49.096168 (+ 488us) server_negotiation.cc:407] Connection header received
0810 06:38:49.096293 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:49.096296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:49.096342 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:49.096418 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:49.096729 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57148: BlockingRecv error: recv got EOF from 127.0.0.1:57148 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:38:49.097610 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 16566
I20260810 06:38:49.100874 17212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:49.098591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59192 (local address 127.31.59.193:45519)
0810 06:38:49.099105 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:49.099111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:49.099135 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:38:49.099188 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:49.099195 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:49.099359 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:49.099469 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:49.100266 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:49.100272 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:49.100281 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:38:49.100299 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:49.100334 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:38:49.100347 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:49.100445 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:49.100526 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 06:38:49.100601 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":252,"threads_started":1}
I20260810 06:38:49.101982 16759 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:49.181267 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 16566
I20260810 06:38:49.202637 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 16770
I20260810 06:38:49.206368 17217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:49.203753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41012 (local address 127.31.59.194:40841)
0810 06:38:49.204335 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:49.204342 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:49.204369 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:38:49.204432 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:49.204438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:49.204659 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:49.204788 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:49.205676 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:49.205682 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:49.205692 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:38:49.205709 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:49.205749 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:38:49.205763 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:49.205867 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:49.205922 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 06:38:49.206024 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":293,"threads_started":1}
I20260810 06:38:49.207605 16972 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:49.286697 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 16770
I20260810 06:38:49.308259 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 16984
I20260810 06:38:49.312285 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:49.309160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42580 (local address 127.31.59.195:37761)
0810 06:38:49.309842 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:49.309849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:49.309879 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:38:49.309941 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:49.309947 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:49.310169 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:49.310335 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:49.311482 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:49.311496 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:49.311507 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:38:49.311524 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:49.311564 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:38:49.311578 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:49.311730 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:49.311861 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 06:38:49.311968 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":356,"threads_started":1}
I20260810 06:38:49.313616 17187 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:49.393684 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 16984
I20260810 06:38:49.416507 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 16413
I20260810 06:38:49.417739 16516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:49.516633 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 16413
2026-08-10T06:38:49Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2326 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:38:49 dist-test-slave-4tdj krb5kdc[17227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:38:49 dist-test-slave-4tdj krb5kdc[17227](info): set up 2 sockets
Aug 10 06:38:49 dist-test-slave-4tdj krb5kdc[17227](info): commencing operation
krb5kdc: starting...
W20260810 06:38:50.757285 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:38:50 dist-test-slave-4tdj krb5kdc[17227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:38:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:38:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:38:51.013204 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:36247
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:45211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:36247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:51.339711 17245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:51.340041 17245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:51.340181 17245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:51.347324 17245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:51.347441 17245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:51.347513 17245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:51.347597 17245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:51.347658 17245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:51.357178 17245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:45211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:36247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:36247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:51.358491 17245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:51.359807 17245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:51.371107 17255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:51.371201 17252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:51.371215 17250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:51.371965 17245 server_base.cc:1034] running on GCE node
I20260810 06:38:51.372694 17245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:51.373996 17245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:51.375202 17245 hybrid_clock.cc:648] HybridClock initialized: now 1786343931375152 us; error 32 us; skew 500 ppm
I20260810 06:38:51.375533 17245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:51 dist-test-slave-4tdj krb5kdc[17227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343931, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:51.381705 17245 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:38:51.383176 17245 webserver.cc:460] Webserver started at http://127.31.59.254:43943/ using document root <none> and password file <none>
I20260810 06:38:51.383824 17245 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:51.383942 17245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:51.384227 17245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:51.387141 17245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "71b58f4cdeb64d9bae334d25dcfbcaa4"
format_stamp: "Formatted at 2026-08-10 06:38:51 on dist-test-slave-4tdj"
server_key: "22960984f58a25856477e4a5bd43bff8"
server_key_iv: "d4d6a2cb441abef87c5aaaca0ebdac63"
server_key_version: "encryptionkey@0"
I20260810 06:38:51.387869 17245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "71b58f4cdeb64d9bae334d25dcfbcaa4"
format_stamp: "Formatted at 2026-08-10 06:38:51 on dist-test-slave-4tdj"
server_key: "22960984f58a25856477e4a5bd43bff8"
server_key_iv: "d4d6a2cb441abef87c5aaaca0ebdac63"
server_key_version: "encryptionkey@0"
I20260810 06:38:51.392830 17245 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:38:51.396293 17263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:51.397562 17245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:38:51.397740 17245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "71b58f4cdeb64d9bae334d25dcfbcaa4"
format_stamp: "Formatted at 2026-08-10 06:38:51 on dist-test-slave-4tdj"
server_key: "22960984f58a25856477e4a5bd43bff8"
server_key_iv: "d4d6a2cb441abef87c5aaaca0ebdac63"
server_key_version: "encryptionkey@0"
I20260810 06:38:51.397936 17245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:51.417094 17245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:51.421476 17245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:51.421772 17245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:51.440469 17245 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:36247
I20260810 06:38:51.440485 17349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:36247 every 8 connection(s)
I20260810 06:38:51.441879 17245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:38:51.445554 17350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:51.449039 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 17245
I20260810 06:38:51.449254 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:38:51.450060 31983 external_mini_cluster.cc:1428] Setting key 08bc23aedfa00faf4e5dce8f976995d2
I20260810 06:38:51.458681 17350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4: Bootstrap starting.
I20260810 06:38:51.462320 17350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:51.463603 17350 log.cc:824] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4: Log is configured to *not* fsync() on all Append() calls
Aug 10 06:38:51 dist-test-slave-4tdj krb5kdc[17227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:38:51.466928 17350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4: No bootstrap required, opened a new log
I20260810 06:38:51.474573 17357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:51.452195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56410 (local address 127.31.59.254:36247)
0810 06:38:51.453027 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:51.453038 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:51.453087 (+ 49us) server_negotiation.cc:407] Connection header received
0810 06:38:51.454798 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:51.454821 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:51.455223 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:51.455574 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:51.457344 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:51.458714 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:51.460475 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:51.461124 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:51.461245 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:51.465999 (+ 4754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:51.466010 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:51.466040 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:51.466086 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:51.469931 (+ 3845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:51.470696 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:51.470705 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:51.470714 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:51.470929 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:51.471313 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:51.471318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:51.471321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:51.471998 (+ 677us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:51.472334 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:51.472704 (+ 370us) server_negotiation.cc:299] Negotiation successful
0810 06:38:51.473138 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":384,"threads_started":1}
I20260810 06:38:51.476794 17350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71b58f4cdeb64d9bae334d25dcfbcaa4" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36247 } }
I20260810 06:38:51.477144 17350 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:51.477262 17350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71b58f4cdeb64d9bae334d25dcfbcaa4, State: Initialized, Role: FOLLOWER
I20260810 06:38:51.477845 17350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71b58f4cdeb64d9bae334d25dcfbcaa4" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36247 } }
I20260810 06:38:51.478055 17350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:38:51.478189 17350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:38:51.478355 17350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:51.480453 17350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71b58f4cdeb64d9bae334d25dcfbcaa4" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36247 } }
I20260810 06:38:51.480911 17350 leader_election.cc:302] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71b58f4cdeb64d9bae334d25dcfbcaa4; no voters:
I20260810 06:38:51.481446 17350 leader_election.cc:288] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:38:51.481592 17361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:51.482218 17361 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4 [term 1 LEADER]: Becoming Leader. State: Replica: 71b58f4cdeb64d9bae334d25dcfbcaa4, State: Running, Role: LEADER
I20260810 06:38:51.482867 17361 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71b58f4cdeb64d9bae334d25dcfbcaa4" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36247 } }
I20260810 06:38:51.483796 17350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:51.486208 17366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71b58f4cdeb64d9bae334d25dcfbcaa4. Latest consensus state: current_term: 1 leader_uuid: "71b58f4cdeb64d9bae334d25dcfbcaa4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71b58f4cdeb64d9bae334d25dcfbcaa4" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36247 } } }
I20260810 06:38:51.486581 17366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:51.485971 17365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71b58f4cdeb64d9bae334d25dcfbcaa4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71b58f4cdeb64d9bae334d25dcfbcaa4" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36247 } } }
I20260810 06:38:51.487502 17365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:51.488749 17379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:38:51.493068 17379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:38:51.501595 17379 catalog_manager.cc:1347] Generated new cluster ID: fcaeaf88b54f4f069593160140e30fa0
I20260810 06:38:51.501739 17379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:38:51.525458 17379 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:38:51.527411 17379 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:38:51.537539 17379 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4: Generated new TSK 0
I20260810 06:38:51.538955 17379 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
I20260810 06:38:51.612444 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36247
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:45211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:51.936362 17403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:51.936643 17403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:51.936758 17403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:51.943874 17403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:51.943984 17403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:51.944095 17403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:38:51.953521 17403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:45211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:51.954712 17403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:51.955933 17403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:51.968508 17412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:51.968571 17411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:51.968513 17416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:51.969168 17403 server_base.cc:1034] running on GCE node
I20260810 06:38:51.969743 17403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:51.970660 17403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:51.971871 17403 hybrid_clock.cc:648] HybridClock initialized: now 1786343931971814 us; error 45 us; skew 500 ppm
I20260810 06:38:51.972177 17403 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:51 dist-test-slave-4tdj krb5kdc[17227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343931, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:51.975178 17403 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:38:51.976521 17403 webserver.cc:460] Webserver started at http://127.31.59.193:41943/ using document root <none> and password file <none>
I20260810 06:38:51.977175 17403 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:51.977280 17403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:51.977557 17403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:51.980360 17403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "cc747864c5604ab8bc0981aad3e67bd2"
format_stamp: "Formatted at 2026-08-10 06:38:51 on dist-test-slave-4tdj"
server_key: "a9e9235c06c038ee0916b0a7c6dbc6b8"
server_key_iv: "24dd280bccfe5a317a110b0db377438d"
server_key_version: "encryptionkey@0"
I20260810 06:38:51.981056 17403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "cc747864c5604ab8bc0981aad3e67bd2"
format_stamp: "Formatted at 2026-08-10 06:38:51 on dist-test-slave-4tdj"
server_key: "a9e9235c06c038ee0916b0a7c6dbc6b8"
server_key_iv: "24dd280bccfe5a317a110b0db377438d"
server_key_version: "encryptionkey@0"
I20260810 06:38:51.986061 17403 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 06:38:51.989518 17427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:51.990825 17403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:38:51.990974 17403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "cc747864c5604ab8bc0981aad3e67bd2"
format_stamp: "Formatted at 2026-08-10 06:38:51 on dist-test-slave-4tdj"
server_key: "a9e9235c06c038ee0916b0a7c6dbc6b8"
server_key_iv: "24dd280bccfe5a317a110b0db377438d"
server_key_version: "encryptionkey@0"
I20260810 06:38:51.991144 17403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:52.013962 17403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:52.018061 17403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:52.018364 17403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:52.019317 17403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:52.020602 17403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:52.020679 17403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:52.020797 17403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:52.020852 17403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:52.050962 17403 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:46715
I20260810 06:38:52.051029 17619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:46715 every 8 connection(s)
I20260810 06:38:52.052155 17403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:38:52.057713 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 17403
I20260810 06:38:52.057891 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:38:52.058732 31983 external_mini_cluster.cc:1428] Setting key 83c309762cea12c4233c9a8decf1ec92
Aug 10 06:38:52 dist-test-slave-4tdj krb5kdc[17227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343931, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:38:52.071635 17623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:52.054486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:55357 (local address 127.31.59.254:36247)
0810 06:38:52.054891 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:52.054896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:52.055877 (+ 981us) server_negotiation.cc:407] Connection header received
0810 06:38:52.057673 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:52.057678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:52.057754 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:52.057868 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:52.059772 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:52.060494 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:52.061991 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:52.062436 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:52.062550 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:52.066101 (+ 3551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:52.066105 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:52.066109 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:52.066140 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:52.068605 (+ 2465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:52.069428 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:52.069431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:52.069434 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:52.069552 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:52.069977 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:52.069979 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:52.069981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:52.070222 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:52.070364 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:52.071087 (+ 723us) server_negotiation.cc:299] Negotiation successful
0810 06:38:52.071296 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":194,"threads_started":1}
I20260810 06:38:52.072521 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:52.055037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36247 (local address 127.31.59.193:55357)
0810 06:38:52.055716 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:52.055760 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:52.057407 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:52.058161 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:52.058185 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:52.058912 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:52.059593 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:52.059610 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:52.060659 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:52.060662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:52.061840 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:52.061851 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:52.062652 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:52.062659 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:52.062882 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:52.063483 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:52.063505 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:52.065896 (+ 2391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:52.068737 (+ 2841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:52.068745 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:52.068759 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:52.069325 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:52.069653 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:52.069658 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:52.069661 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:52.069888 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:52.070372 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:52.070379 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:52.070746 (+ 367us) client_negotiation.cc:769] Sending connection context
0810 06:38:52.071036 (+ 290us) client_negotiation.cc:240] Negotiation successful
0810 06:38:52.071454 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":257,"threads_started":1}
I20260810 06:38:52.073982 17620 heartbeater.cc:347] Connected to a master server at 127.31.59.254:36247
I20260810 06:38:52.074266 17620 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:52.074986 17620 heartbeater.cc:511] Master 127.31.59.254:36247 requested a full tablet report, sending...
I20260810 06:38:52.076965 17295 ts_manager.cc:194] Registered new tserver with Master: cc747864c5604ab8bc0981aad3e67bd2 (127.31.59.193:46715)
I20260810 06:38:52.078441 17295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:55357
WARNING: no policy specified for kudu/127.31.59.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
I20260810 06:38:52.115238 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36247
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:45211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:52.432240 17636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:52.432512 17636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:52.432658 17636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:52.439611 17636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:52.439749 17636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:52.439904 17636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:38:52.449255 17636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:45211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:52.450456 17636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:52.451701 17636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:52.463786 17649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:52.463863 17651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:52.463976 17654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:52.464852 17636 server_base.cc:1034] running on GCE node
I20260810 06:38:52.465446 17636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:52.466367 17636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:52.467566 17636 hybrid_clock.cc:648] HybridClock initialized: now 1786343932467523 us; error 40 us; skew 500 ppm
I20260810 06:38:52.467856 17636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:52 dist-test-slave-4tdj krb5kdc[17227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343932, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:52.470887 17636 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:38:52.472193 17636 webserver.cc:460] Webserver started at http://127.31.59.194:36173/ using document root <none> and password file <none>
I20260810 06:38:52.472833 17636 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:52.472944 17636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:52.473229 17636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:52.476007 17636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "f006d9f3e4ff44a9bcf13434ab76d1b4"
format_stamp: "Formatted at 2026-08-10 06:38:52 on dist-test-slave-4tdj"
server_key: "cea0e0cf264d7b40367f0ef3a53d2f38"
server_key_iv: "733dd745a63148a5e1d581b39b306cdb"
server_key_version: "encryptionkey@0"
I20260810 06:38:52.476747 17636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "f006d9f3e4ff44a9bcf13434ab76d1b4"
format_stamp: "Formatted at 2026-08-10 06:38:52 on dist-test-slave-4tdj"
server_key: "cea0e0cf264d7b40367f0ef3a53d2f38"
server_key_iv: "733dd745a63148a5e1d581b39b306cdb"
server_key_version: "encryptionkey@0"
I20260810 06:38:52.481492 17636 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:38:52.484822 17665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:52.486120 17636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 06:38:52.486249 17636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "f006d9f3e4ff44a9bcf13434ab76d1b4"
format_stamp: "Formatted at 2026-08-10 06:38:52 on dist-test-slave-4tdj"
server_key: "cea0e0cf264d7b40367f0ef3a53d2f38"
server_key_iv: "733dd745a63148a5e1d581b39b306cdb"
server_key_version: "encryptionkey@0"
I20260810 06:38:52.486483 17636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:52.502671 17636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:52.506279 17636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:52.506597 17636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:52.507515 17636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:52.508817 17636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:52.508888 17636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:52.508999 17636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:52.509049 17636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:52.540217 17636 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:42501
I20260810 06:38:52.540241 17858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:42501 every 8 connection(s)
I20260810 06:38:52.541428 17636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:38:52.547688 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 17636
I20260810 06:38:52.547823 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:38:52.548252 31983 external_mini_cluster.cc:1428] Setting key e48acae50c67516a1c5524d98f170512
Aug 10 06:38:52 dist-test-slave-4tdj krb5kdc[17227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343932, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:38:52.560570 17623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:52.543918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:48073 (local address 127.31.59.254:36247)
0810 06:38:52.544107 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:52.544112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:52.545274 (+ 1162us) server_negotiation.cc:407] Connection header received
0810 06:38:52.547123 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:52.547128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:52.547205 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:52.547294 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:52.548892 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:52.549695 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:52.551378 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:52.551791 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:52.551899 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:52.555165 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:52.555170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:52.555174 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:52.555207 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:52.557566 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:52.558263 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:52.558267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:52.558270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:52.558437 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:52.558871 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:52.558876 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:52.558879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:52.559150 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:52.559302 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:52.559983 (+ 681us) server_negotiation.cc:299] Negotiation successful
0810 06:38:52.560223 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:38:52.561447 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:52.544402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36247 (local address 127.31.59.194:48073)
0810 06:38:52.545101 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:52.545146 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:52.546849 (+ 1703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:52.547554 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:52.547562 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:52.548118 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:52.548709 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:52.548734 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:52.549855 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:52.549861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:52.551229 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:52.551241 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:52.552010 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:52.552018 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:52.552230 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:52.552844 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:52.552866 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:52.554945 (+ 2079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:52.557707 (+ 2762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:52.557715 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:52.557731 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:52.558132 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:52.558556 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:52.558561 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:52.558564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:52.558747 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:52.559285 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:52.559292 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:52.559624 (+ 332us) client_negotiation.cc:769] Sending connection context
0810 06:38:52.559860 (+ 236us) client_negotiation.cc:240] Negotiation successful
0810 06:38:52.560359 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":261,"threads_started":1}
I20260810 06:38:52.562788 17859 heartbeater.cc:347] Connected to a master server at 127.31.59.254:36247
I20260810 06:38:52.563105 17859 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:52.563828 17859 heartbeater.cc:511] Master 127.31.59.254:36247 requested a full tablet report, sending...
I20260810 06:38:52.565256 17295 ts_manager.cc:194] Registered new tserver with Master: f006d9f3e4ff44a9bcf13434ab76d1b4 (127.31.59.194:42501)
I20260810 06:38:52.566043 17295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:48073
WARNING: no policy specified for kudu/127.31.59.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
I20260810 06:38:52.605176 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36247
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:45211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:52.925419 17869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:52.925681 17869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:52.925776 17869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:52.932693 17869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:52.932790 17869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:52.932907 17869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:38:52.942118 17869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:45211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:38:52.943666 17869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:52.945142 17869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:52.957502 17879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:52.957497 17883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:52.957612 17878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:52.958510 17869 server_base.cc:1034] running on GCE node
I20260810 06:38:52.959064 17869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:52.959913 17869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:52.961109 17869 hybrid_clock.cc:648] HybridClock initialized: now 1786343932961067 us; error 30 us; skew 500 ppm
I20260810 06:38:52.961388 17869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:38:52 dist-test-slave-4tdj krb5kdc[17227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343932, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:52.967953 17869 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:38:52.969401 17869 webserver.cc:460] Webserver started at http://127.31.59.195:41635/ using document root <none> and password file <none>
I20260810 06:38:52.970116 17869 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:52.970248 17869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:52.970570 17869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:52.973325 17869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "fd16bec357f240d7b1a2e9e4e9155785"
format_stamp: "Formatted at 2026-08-10 06:38:52 on dist-test-slave-4tdj"
server_key: "9c576bf68aa58b9d5be51562f585989b"
server_key_iv: "85af8fd54455ff866acf6725fe399330"
server_key_version: "encryptionkey@0"
I20260810 06:38:52.974014 17869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "fd16bec357f240d7b1a2e9e4e9155785"
format_stamp: "Formatted at 2026-08-10 06:38:52 on dist-test-slave-4tdj"
server_key: "9c576bf68aa58b9d5be51562f585989b"
server_key_iv: "85af8fd54455ff866acf6725fe399330"
server_key_version: "encryptionkey@0"
I20260810 06:38:52.978868 17869 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 06:38:52.982059 17893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:52.983318 17869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:38:52.983467 17869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "fd16bec357f240d7b1a2e9e4e9155785"
format_stamp: "Formatted at 2026-08-10 06:38:52 on dist-test-slave-4tdj"
server_key: "9c576bf68aa58b9d5be51562f585989b"
server_key_iv: "85af8fd54455ff866acf6725fe399330"
server_key_version: "encryptionkey@0"
I20260810 06:38:52.983647 17869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:53.005792 17869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:53.009804 17869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:53.010082 17869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:53.011068 17869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:53.012363 17869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:53.012439 17869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:53.012529 17869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:53.012588 17869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:53.046119 17869 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:45251
I20260810 06:38:53.046203 18091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:45251 every 8 connection(s)
I20260810 06:38:53.047376 17869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:38:53.057328 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 17869
I20260810 06:38:53.057511 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:38:53.058087 31983 external_mini_cluster.cc:1428] Setting key b67d41dca08fa1b771cf3f48dfafb2b1
Aug 10 06:38:53 dist-test-slave-4tdj krb5kdc[17227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343932, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:38:53.069591 17623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:53.049796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:38479 (local address 127.31.59.254:36247)
0810 06:38:53.049983 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:53.049987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:53.051339 (+ 1352us) server_negotiation.cc:407] Connection header received
0810 06:38:53.053785 (+ 2446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:53.053789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:53.053894 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:53.054015 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:53.055897 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.056675 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.059019 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.059651 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.059790 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:53.063517 (+ 3727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:53.063522 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:53.063525 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:53.063558 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:53.066278 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:53.067054 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:53.067061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:53.067063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:53.067212 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:53.067713 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:53.067717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:53.067719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:53.068035 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:53.068183 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:53.068912 (+ 729us) server_negotiation.cc:299] Negotiation successful
0810 06:38:53.069162 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:38:53.070574 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:53.050159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36247 (local address 127.31.59.195:38479)
0810 06:38:53.051145 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:53.051195 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:53.053489 (+ 2294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:53.054266 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:53.054277 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:53.054965 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:53.055699 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:53.055726 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.056831 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.056835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:53.058821 (+ 1986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:53.058836 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.059908 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.059917 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:53.060165 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:53.060850 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:53.060874 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:53.063286 (+ 2412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:53.066450 (+ 3164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:53.066461 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:53.066477 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:53.066906 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:53.067348 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:53.067354 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:53.067358 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:53.067562 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:53.068178 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:53.068186 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:53.068516 (+ 330us) client_negotiation.cc:769] Sending connection context
0810 06:38:53.068766 (+ 250us) client_negotiation.cc:240] Negotiation successful
0810 06:38:53.069285 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":339,"threads_started":1}
I20260810 06:38:53.071982 18095 heartbeater.cc:347] Connected to a master server at 127.31.59.254:36247
I20260810 06:38:53.072299 18095 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:53.073087 18095 heartbeater.cc:511] Master 127.31.59.254:36247 requested a full tablet report, sending...
I20260810 06:38:53.074553 17295 ts_manager.cc:194] Registered new tserver with Master: fd16bec357f240d7b1a2e9e4e9155785 (127.31.59.195:45251)
I20260810 06:38:53.075328 17295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:38479
I20260810 06:38:53.081221 17620 heartbeater.cc:503] Master 127.31.59.254:36247 was elected leader, sending a full tablet report...
I20260810 06:38:53.085564 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:38:53.102869 17623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:53.090118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56412 (local address 127.31.59.254:36247)
0810 06:38:53.090332 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:53.090339 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:53.090775 (+ 436us) server_negotiation.cc:407] Connection header received
0810 06:38:53.090916 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:53.090920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:53.090978 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:53.091058 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:53.092676 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.093388 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.094771 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.095203 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.095302 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:53.097592 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:53.097597 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:53.097600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:53.097629 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:53.100126 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:53.100899 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:53.100905 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:53.100908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:53.101116 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:53.101580 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:53.101586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:53.101589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:53.101941 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:53.102151 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:53.102473 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 06:38:53.102640 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 06:38:53.106741 17295 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:38:53.113358 17295 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:38:53.150929 18118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:53.125746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:45251 (local address 127.0.0.1:34528)
0810 06:38:53.143313 (+ 17567us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:53.143331 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:53.143557 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:53.144072 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:53.144077 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:53.144100 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:53.144494 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:53.144506 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.146386 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.146392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:53.148168 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:53.148181 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.149900 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.149907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:53.150102 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:53.150157 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 06:38:53.150274 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 06:38:53.150501 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17434,"mutex_wait_us":83,"thread_start_us":17294,"threads_started":1}
I20260810 06:38:53.151262 18116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:53.125136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35040 (local address 127.31.59.194:42501)
0810 06:38:53.142732 (+ 17596us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:53.142742 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:53.142776 (+ 34us) server_negotiation.cc:407] Connection header received
0810 06:38:53.142891 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:53.142899 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:53.143173 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:53.144106 (+ 933us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:53.144323 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.145834 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.148745 (+ 2911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.150165 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.150334 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:53.150516 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:53.150791 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 06:38:53.150957 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17428,"thread_start_us":17273,"threads_started":1}
I20260810 06:38:53.151746 18121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:53.126399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34528 (local address 127.31.59.195:45251)
0810 06:38:53.141542 (+ 15143us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:53.141552 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:53.143438 (+ 1886us) server_negotiation.cc:407] Connection header received
0810 06:38:53.143717 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:53.143725 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:53.143892 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:53.144061 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:53.144693 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.146213 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.148329 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.149710 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.149839 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:53.150069 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:53.151190 (+ 1121us) server_negotiation.cc:299] Negotiation successful
0810 06:38:53.151437 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14965,"thread_start_us":14804,"threads_started":1}
I20260810 06:38:53.151855 18119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:53.125871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46624 (local address 127.31.59.193:46715)
0810 06:38:53.143695 (+ 17824us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:53.143702 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:53.143731 (+ 29us) server_negotiation.cc:407] Connection header received
0810 06:38:53.143834 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:53.143841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:53.144058 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:53.144944 (+ 886us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:53.145167 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.146508 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.148925 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.150274 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.150428 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:53.150638 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:53.151380 (+ 742us) server_negotiation.cc:299] Negotiation successful
0810 06:38:53.151531 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17662,"thread_start_us":17502,"threads_started":1}
I20260810 06:38:53.151238 18112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:53.125014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:42501 (local address 127.0.0.1:35040)
0810 06:38:53.139976 (+ 14962us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:53.140024 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:53.140238 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:53.143429 (+ 3191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:53.143437 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:53.143468 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:53.143901 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:53.143914 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.146017 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.146022 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:53.148558 (+ 2536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:53.148572 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.150408 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.150415 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:53.150593 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:53.150630 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 06:38:53.150725 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 06:38:53.150856 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14658,"mutex_wait_us":50,"thread_start_us":15398,"threads_started":1}
I20260810 06:38:53.152002 18117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:53.125169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:46715 (local address 127.0.0.1:46624)
0810 06:38:53.141982 (+ 16813us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:53.142001 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:53.142135 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:53.144291 (+ 2156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:53.144296 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:53.144319 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:53.144753 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:53.144767 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.146675 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.146680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:53.148796 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:53.148810 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.150790 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.150797 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:53.150956 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:53.150990 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 06:38:53.151284 (+ 294us) client_negotiation.cc:240] Negotiation successful
0810 06:38:53.151503 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16681,"mutex_wait_us":1,"thread_start_us":16545,"threads_started":1}
I20260810 06:38:53.156170 17750 tablet_service.cc:1467] Processing CreateTablet for tablet 00149e1f1069409ea374e99554b81c85 (DEFAULT_TABLE table=test-table [id=c284ce4f4a484f4991802b342e324d5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:53.156170 17990 tablet_service.cc:1467] Processing CreateTablet for tablet 00149e1f1069409ea374e99554b81c85 (DEFAULT_TABLE table=test-table [id=c284ce4f4a484f4991802b342e324d5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:53.156378 17512 tablet_service.cc:1467] Processing CreateTablet for tablet 00149e1f1069409ea374e99554b81c85 (DEFAULT_TABLE table=test-table [id=c284ce4f4a484f4991802b342e324d5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:53.157586 17750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00149e1f1069409ea374e99554b81c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:53.157580 17990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00149e1f1069409ea374e99554b81c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:53.157579 17512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00149e1f1069409ea374e99554b81c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:53.168737 18122 tablet_bootstrap.cc:492] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2: Bootstrap starting.
I20260810 06:38:53.171994 18126 tablet_bootstrap.cc:492] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785: Bootstrap starting.
I20260810 06:38:53.173277 18122 tablet_bootstrap.cc:654] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:53.174614 18122 log.cc:824] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:53.175040 18124 tablet_bootstrap.cc:492] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4: Bootstrap starting.
I20260810 06:38:53.175374 18126 tablet_bootstrap.cc:654] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:53.176414 18126 log.cc:824] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:53.178103 18124 tablet_bootstrap.cc:654] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:53.178215 18122 tablet_bootstrap.cc:492] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2: No bootstrap required, opened a new log
I20260810 06:38:53.178540 18122 ts_tablet_manager.cc:1397] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 06:38:53.179203 18124 log.cc:824] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:53.181550 18124 tablet_bootstrap.cc:492] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4: No bootstrap required, opened a new log
I20260810 06:38:53.181782 18124 ts_tablet_manager.cc:1397] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 06:38:53.184835 18126 tablet_bootstrap.cc:492] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785: No bootstrap required, opened a new log
I20260810 06:38:53.185122 18126 ts_tablet_manager.cc:1397] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.008s
I20260810 06:38:53.189296 18122 raft_consensus.cc:348] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f006d9f3e4ff44a9bcf13434ab76d1b4" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42501 } } peers { permanent_uuid: "cc747864c5604ab8bc0981aad3e67bd2" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46715 } } peers { permanent_uuid: "fd16bec357f240d7b1a2e9e4e9155785" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45251 } }
I20260810 06:38:53.189826 18122 raft_consensus.cc:374] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:53.189564 18124 raft_consensus.cc:348] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f006d9f3e4ff44a9bcf13434ab76d1b4" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42501 } } peers { permanent_uuid: "cc747864c5604ab8bc0981aad3e67bd2" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46715 } } peers { permanent_uuid: "fd16bec357f240d7b1a2e9e4e9155785" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45251 } }
I20260810 06:38:53.189918 18122 raft_consensus.cc:729] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc747864c5604ab8bc0981aad3e67bd2, State: Initialized, Role: FOLLOWER
I20260810 06:38:53.190026 18124 raft_consensus.cc:374] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:53.190142 18124 raft_consensus.cc:729] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f006d9f3e4ff44a9bcf13434ab76d1b4, State: Initialized, Role: FOLLOWER
I20260810 06:38:53.190685 18122 consensus_queue.cc:260] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f006d9f3e4ff44a9bcf13434ab76d1b4" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42501 } } peers { permanent_uuid: "cc747864c5604ab8bc0981aad3e67bd2" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46715 } } peers { permanent_uuid: "fd16bec357f240d7b1a2e9e4e9155785" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45251 } }
I20260810 06:38:53.190847 18124 consensus_queue.cc:260] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f006d9f3e4ff44a9bcf13434ab76d1b4" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42501 } } peers { permanent_uuid: "cc747864c5604ab8bc0981aad3e67bd2" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46715 } } peers { permanent_uuid: "fd16bec357f240d7b1a2e9e4e9155785" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45251 } }
I20260810 06:38:53.191963 17859 heartbeater.cc:503] Master 127.31.59.254:36247 was elected leader, sending a full tablet report...
I20260810 06:38:53.192408 18124 ts_tablet_manager.cc:1428] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260810 06:38:53.195289 18126 raft_consensus.cc:348] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f006d9f3e4ff44a9bcf13434ab76d1b4" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42501 } } peers { permanent_uuid: "cc747864c5604ab8bc0981aad3e67bd2" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46715 } } peers { permanent_uuid: "fd16bec357f240d7b1a2e9e4e9155785" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45251 } }
I20260810 06:38:53.195720 18126 raft_consensus.cc:374] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:53.195815 18126 raft_consensus.cc:729] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd16bec357f240d7b1a2e9e4e9155785, State: Initialized, Role: FOLLOWER
I20260810 06:38:53.196421 18126 consensus_queue.cc:260] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f006d9f3e4ff44a9bcf13434ab76d1b4" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42501 } } peers { permanent_uuid: "cc747864c5604ab8bc0981aad3e67bd2" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46715 } } peers { permanent_uuid: "fd16bec357f240d7b1a2e9e4e9155785" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45251 } }
I20260810 06:38:53.197518 18095 heartbeater.cc:503] Master 127.31.59.254:36247 was elected leader, sending a full tablet report...
I20260810 06:38:53.198184 18126 ts_tablet_manager.cc:1428] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260810 06:38:53.199441 18122 ts_tablet_manager.cc:1428] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2: Time spent starting tablet: real 0.021s user 0.011s sys 0.008s
I20260810 06:38:53.280068 18134 raft_consensus.cc:482] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:53.280377 18134 raft_consensus.cc:504] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f006d9f3e4ff44a9bcf13434ab76d1b4" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42501 } } peers { permanent_uuid: "cc747864c5604ab8bc0981aad3e67bd2" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46715 } } peers { permanent_uuid: "fd16bec357f240d7b1a2e9e4e9155785" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45251 } }
I20260810 06:38:53.282199 18134 leader_election.cc:288] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc747864c5604ab8bc0981aad3e67bd2 (127.31.59.193:46715), fd16bec357f240d7b1a2e9e4e9155785 (127.31.59.195:45251)
I20260810 06:38:53.291064 18119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:53.282747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:39537 (local address 127.31.59.193:46715)
0810 06:38:53.283086 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:53.283093 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:53.283465 (+ 372us) server_negotiation.cc:407] Connection header received
0810 06:38:53.283716 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:53.283721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:53.283801 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:53.283899 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:53.284564 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.285733 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.288659 (+ 2926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.289907 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.290069 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:53.290141 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:53.290692 (+ 551us) server_negotiation.cc:299] Negotiation successful
0810 06:38:53.290820 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260810 06:38:53.291067 18121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:53.282839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:57247 (local address 127.31.59.195:45251)
0810 06:38:53.283243 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:53.283250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:53.283725 (+ 475us) server_negotiation.cc:407] Connection header received
0810 06:38:53.283833 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:53.283838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:53.283912 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:53.283995 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:53.284642 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.285733 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.288663 (+ 2930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.289912 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.290073 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:53.290141 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:53.290692 (+ 551us) server_negotiation.cc:299] Negotiation successful
0810 06:38:53.290820 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276}
I20260810 06:38:53.291297 18138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:53.282459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:46715 (local address 127.31.59.194:39537)
0810 06:38:53.283354 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:53.283379 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:53.283564 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:53.283927 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:53.283931 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:53.283952 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:53.284373 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:53.284389 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.285879 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.285885 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:53.288518 (+ 2633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:53.288528 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.290161 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.290167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:53.290505 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:53.290547 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 06:38:53.290647 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 06:38:53.290782 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"mutex_wait_us":165,"thread_start_us":870,"threads_started":1}
I20260810 06:38:53.291376 18137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:53.282599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:45251 (local address 127.31.59.194:57247)
0810 06:38:53.283618 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:53.283634 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:53.283738 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:53.284049 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:53.284054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:53.284090 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:53.284490 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:53.284500 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.285882 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.285886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:53.288499 (+ 2613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:53.288519 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.290149 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.290156 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:53.290530 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:53.290559 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 06:38:53.290689 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 06:38:53.290791 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"mutex_wait_us":243,"thread_start_us":366,"threads_started":1}
I20260810 06:38:53.292388 18019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00149e1f1069409ea374e99554b81c85" candidate_uuid: "f006d9f3e4ff44a9bcf13434ab76d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd16bec357f240d7b1a2e9e4e9155785" is_pre_election: true
I20260810 06:38:53.292408 17544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00149e1f1069409ea374e99554b81c85" candidate_uuid: "f006d9f3e4ff44a9bcf13434ab76d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc747864c5604ab8bc0981aad3e67bd2" is_pre_election: true
I20260810 06:38:53.292928 18019 raft_consensus.cc:2454] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f006d9f3e4ff44a9bcf13434ab76d1b4 in term 0.
I20260810 06:38:53.293026 17544 raft_consensus.cc:2454] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f006d9f3e4ff44a9bcf13434ab76d1b4 in term 0.
W20260810 06:38:53.293361 17860 tablet.cc:2369] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:38:53.293601 17671 leader_election.cc:302] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f006d9f3e4ff44a9bcf13434ab76d1b4, fd16bec357f240d7b1a2e9e4e9155785; no voters:
I20260810 06:38:53.293994 18134 raft_consensus.cc:2781] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:38:53.294153 18134 raft_consensus.cc:482] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:38:53.294320 18134 raft_consensus.cc:3037] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:53.296582 18134 raft_consensus.cc:504] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f006d9f3e4ff44a9bcf13434ab76d1b4" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42501 } } peers { permanent_uuid: "cc747864c5604ab8bc0981aad3e67bd2" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46715 } } peers { permanent_uuid: "fd16bec357f240d7b1a2e9e4e9155785" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45251 } }
I20260810 06:38:53.297140 18134 leader_election.cc:288] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [CANDIDATE]: Term 1 election: Requested vote from peers cc747864c5604ab8bc0981aad3e67bd2 (127.31.59.193:46715), fd16bec357f240d7b1a2e9e4e9155785 (127.31.59.195:45251)
I20260810 06:38:53.297657 17544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00149e1f1069409ea374e99554b81c85" candidate_uuid: "f006d9f3e4ff44a9bcf13434ab76d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc747864c5604ab8bc0981aad3e67bd2"
I20260810 06:38:53.297657 18019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00149e1f1069409ea374e99554b81c85" candidate_uuid: "f006d9f3e4ff44a9bcf13434ab76d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd16bec357f240d7b1a2e9e4e9155785"
I20260810 06:38:53.297863 17544 raft_consensus.cc:3037] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:53.297863 18019 raft_consensus.cc:3037] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785 [term 0 FOLLOWER]: Advancing to term 1
W20260810 06:38:53.298877 18096 tablet.cc:2369] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:38:53.300979 18019 raft_consensus.cc:2454] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f006d9f3e4ff44a9bcf13434ab76d1b4 in term 1.
I20260810 06:38:53.301003 17544 raft_consensus.cc:2454] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f006d9f3e4ff44a9bcf13434ab76d1b4 in term 1.
I20260810 06:38:53.301441 17667 leader_election.cc:302] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc747864c5604ab8bc0981aad3e67bd2, f006d9f3e4ff44a9bcf13434ab76d1b4; no voters:
I20260810 06:38:53.301776 18134 raft_consensus.cc:2781] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:53.302202 18134 raft_consensus.cc:686] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [term 1 LEADER]: Becoming Leader. State: Replica: f006d9f3e4ff44a9bcf13434ab76d1b4, State: Running, Role: LEADER
I20260810 06:38:53.302726 18134 consensus_queue.cc:237] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f006d9f3e4ff44a9bcf13434ab76d1b4" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42501 } } peers { permanent_uuid: "cc747864c5604ab8bc0981aad3e67bd2" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46715 } } peers { permanent_uuid: "fd16bec357f240d7b1a2e9e4e9155785" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45251 } }
W20260810 06:38:53.304311 17622 tablet.cc:2369] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:38:53.307713 17292 catalog_manager.cc:5495] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f006d9f3e4ff44a9bcf13434ab76d1b4 (127.31.59.194). New cstate: current_term: 1 leader_uuid: "f006d9f3e4ff44a9bcf13434ab76d1b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f006d9f3e4ff44a9bcf13434ab76d1b4" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc747864c5604ab8bc0981aad3e67bd2" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd16bec357f240d7b1a2e9e4e9155785" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45251 } health_report { overall_health: UNKNOWN } } }
I20260810 06:38:53.342978 18116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:53.337676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35048 (local address 127.31.59.194:42501)
0810 06:38:53.337900 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:53.337906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:53.337991 (+ 85us) server_negotiation.cc:407] Connection header received
0810 06:38:53.338221 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:53.338225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:53.338322 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:53.338447 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:53.338830 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.339566 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.340910 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.341309 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.341419 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:53.341877 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:53.342381 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:38:53.342520 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:53.342640 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 06:38:53.342722 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:38:53.350981 17544 raft_consensus.cc:1261] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2 [term 1 FOLLOWER]: Refusing update from remote peer f006d9f3e4ff44a9bcf13434ab76d1b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:53.350981 18019 raft_consensus.cc:1261] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785 [term 1 FOLLOWER]: Refusing update from remote peer f006d9f3e4ff44a9bcf13434ab76d1b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:53.351954 18139 consensus_queue.cc:1035] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc747864c5604ab8bc0981aad3e67bd2" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:53.352279 18134 consensus_queue.cc:1035] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd16bec357f240d7b1a2e9e4e9155785" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 45251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:53.362347 18144 mvcc.cc:205] Tried to move back new op lower bound from 7316864750995132416 to 7316864750811279360. Current Snapshot: MvccSnapshot[applied={T|T < 7316864750995132416}]
I20260810 06:38:53.374212 17292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:53.374545 17292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:53.378039 17292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 71b58f4cdeb64d9bae334d25dcfbcaa4: Sending DeleteTablet for 3 replicas of tablet 00149e1f1069409ea374e99554b81c85
I20260810 06:38:53.379097 17750 tablet_service.cc:1514] Processing DeleteTablet for tablet 00149e1f1069409ea374e99554b81c85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:53 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:35040
I20260810 06:38:53.379519 17990 tablet_service.cc:1514] Processing DeleteTablet for tablet 00149e1f1069409ea374e99554b81c85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:53 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:34528
I20260810 06:38:53.379524 17512 tablet_service.cc:1514] Processing DeleteTablet for tablet 00149e1f1069409ea374e99554b81c85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:53 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:46624
I20260810 06:38:53.383096 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 17403
I20260810 06:38:53.393186 18167 tablet_replica.cc:330] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2: stopping tablet replica
I20260810 06:38:53.394279 18167 raft_consensus.cc:2229] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:53.395475 18167 raft_consensus.cc:2258] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:53.396430 18165 tablet_replica.cc:330] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785: stopping tablet replica
I20260810 06:38:53.397547 18165 raft_consensus.cc:2229] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:53.398111 18165 raft_consensus.cc:2258] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:53.398412 18164 tablet_replica.cc:330] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4: stopping tablet replica
I20260810 06:38:53.398828 18164 raft_consensus.cc:2229] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:38:53.399030 18167 ts_tablet_manager.cc:1905] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:53.399564 18164 raft_consensus.cc:2258] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:53.400630 18165 ts_tablet_manager.cc:1905] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 06:38:53 dist-test-slave-4tdj krb5kdc[17227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:38:53.406152 18167 ts_tablet_manager.cc:1918] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:53.406368 18167 log.cc:1196] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/00149e1f1069409ea374e99554b81c85
I20260810 06:38:53.406908 18167 ts_tablet_manager.cc:1939] T 00149e1f1069409ea374e99554b81c85 P cc747864c5604ab8bc0981aad3e67bd2: Deleting consensus metadata
I20260810 06:38:53.407912 18165 ts_tablet_manager.cc:1918] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:53.408149 18165 log.cc:1196] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/00149e1f1069409ea374e99554b81c85
I20260810 06:38:53.408789 18165 ts_tablet_manager.cc:1939] T 00149e1f1069409ea374e99554b81c85 P fd16bec357f240d7b1a2e9e4e9155785: Deleting consensus metadata
I20260810 06:38:53.408776 17268 catalog_manager.cc:4841] TS cc747864c5604ab8bc0981aad3e67bd2 (127.31.59.193:46715): tablet 00149e1f1069409ea374e99554b81c85 (table test-table [id=c284ce4f4a484f4991802b342e324d5b]) successfully deleted
I20260810 06:38:53.410362 18164 ts_tablet_manager.cc:1905] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:53.410689 17271 catalog_manager.cc:4841] TS fd16bec357f240d7b1a2e9e4e9155785 (127.31.59.195:45251): tablet 00149e1f1069409ea374e99554b81c85 (table test-table [id=c284ce4f4a484f4991802b342e324d5b]) successfully deleted
I20260810 06:38:53.412878 18119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:53.384172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46630 (local address 127.31.59.193:46715)
0810 06:38:53.384365 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:53.384374 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:53.388929 (+ 4555us) server_negotiation.cc:407] Connection header received
0810 06:38:53.389612 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:53.389620 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:53.389725 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:53.389809 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:53.393007 (+ 3198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.394224 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.396331 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.396995 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.397143 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:53.404553 (+ 7410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:53.404570 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:53.404587 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:53.404633 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:53.409187 (+ 4554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:53.410082 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:53.410103 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:53.410107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:53.410279 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:53.411234 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:53.411239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:53.411242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:53.411612 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:53.411899 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:53.412264 (+ 365us) server_negotiation.cc:299] Negotiation successful
0810 06:38:53.412477 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:38:53.413537 17614 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:53.417666 18164 ts_tablet_manager.cc:1918] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:53.417829 18164 log.cc:1196] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/00149e1f1069409ea374e99554b81c85
I20260810 06:38:53.418407 18164 ts_tablet_manager.cc:1939] T 00149e1f1069409ea374e99554b81c85 P f006d9f3e4ff44a9bcf13434ab76d1b4: Deleting consensus metadata
I20260810 06:38:53.420130 17266 catalog_manager.cc:4841] TS f006d9f3e4ff44a9bcf13434ab76d1b4 (127.31.59.194:42501): tablet 00149e1f1069409ea374e99554b81c85 (table test-table [id=c284ce4f4a484f4991802b342e324d5b]) successfully deleted
I20260810 06:38:53.502667 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 17403
I20260810 06:38:53.528926 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 17636
Aug 10 06:38:53 dist-test-slave-4tdj krb5kdc[17227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.194@KRBTEST.COM
I20260810 06:38:53.547734 18116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:53.529791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35060 (local address 127.31.59.194:42501)
0810 06:38:53.529984 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:53.529989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:53.530014 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:38:53.530211 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:53.530215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:53.530332 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:53.530447 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:53.532018 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.532833 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.534666 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.535129 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.535238 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:53.540340 (+ 5102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:53.540353 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:53.540371 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:53.540424 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:53.544482 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:53.545294 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:53.545303 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:53.545307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:53.545490 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:53.545897 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:53.545902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:53.545906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:53.546405 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:53.546715 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:53.546985 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 06:38:53.547208 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 06:38:53.548441 17851 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:53.660354 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 17636
I20260810 06:38:53.687325 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 17869
Aug 10 06:38:53 dist-test-slave-4tdj krb5kdc[17227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.195@KRBTEST.COM
I20260810 06:38:53.701714 18121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:53.688117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34538 (local address 127.31.59.195:45251)
0810 06:38:53.688312 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:53.688319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:53.688344 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:38:53.688455 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:53.688459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:53.688536 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:53.688641 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:53.689898 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.690742 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.692047 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:53.692473 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:53.692579 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:53.696670 (+ 4091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:53.696685 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:53.696700 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:53.696737 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:53.699265 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:53.699843 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:53.699849 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:53.699852 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:53.699973 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:53.700395 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:53.700398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:53.700400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:53.700720 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:53.700967 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:53.701258 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 06:38:53.701411 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:38:53.702486 18083 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:53.789562 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 17869
I20260810 06:38:53.812189 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 17245
I20260810 06:38:53.813289 17345 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:38:53.921712 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 17245
2026-08-10T06:38:53Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4409 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:38:54 dist-test-slave-4tdj krb5kdc[18178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:38:54 dist-test-slave-4tdj krb5kdc[18178](info): set up 2 sockets
Aug 10 06:38:54 dist-test-slave-4tdj krb5kdc[18178](info): commencing operation
krb5kdc: starting...
W20260810 06:38:55.166918 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.005s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:38:55 dist-test-slave-4tdj krb5kdc[18178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:38:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:38:55Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.59.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.59.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.59.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/rangeradmin_127.31.59.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.59.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/rangeradmin_127.31.59.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.59.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.59.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.59.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/rangerlookup_127.31.59.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.59.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/rangerlookup_127.31.59.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.59.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.59.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.59.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/HTTP_127.31.59.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.59.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/HTTP_127.31.59.212@KRBTEST.COM.keytab.
I20260810 06:38:55.505072 31983 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskq68xXZ/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/postgres -l logfile start
2026-08-10 06:38:57.881 UTC [18229] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 06:38:57.881 UTC [18229] LOG: listening on IPv4 address "127.31.59.212", port 54591
2026-08-10 06:38:57.885 UTC [18229] LOG: listening on Unix socket "/tmp/.s.PGSQL.54591"
2026-08-10 06:38:57.906 UTC [18232] LOG: database system was shut down at 2026-08-10 06:38:56 UTC
2026-08-10 06:38:57.910 UTC [18229] LOG: database system is ready to accept connections
I20260810 06:38:59.007388 31983 mini_postgres.cc:96] Postgres bound to 54591
2026-08-10 06:38:59.030 UTC [18243] FATAL: database "slave" does not exist
127.31.59.212:54591 - accepting connections
I20260810 06:38:59.030833 31983 mini_ranger.cc:161] Starting Ranger...
I20260810 06:38:59.058656 31983 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 06:38:59.534121 31983 mini_ranger.cc:87] Created ranger Postgres database
I20260810 06:38:59.534269 31983 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ranger-admin
2026-08-10 06:38:59,918 [I] DB FLAVOR :POSTGRES
2026-08-10 06:38:59,919 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:38:59,919 [I] Checking connection..
2026-08-10 06:38:59,919 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 06:39:00,190 [I] Checking connection passed.
2026-08-10 06:39:00,190 [I] --------- Verifying version history table ---------
2026-08-10 06:39:00,191 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:39:00,452 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:39:00,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:39:00,686 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:39:00,686 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 06:39:00,687 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 06:39:00.920 UTC [18351] WARNING: there is no transaction in progress
2026-08-10 06:39:00,926 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 06:39:00,927 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:39:01,167 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 06:39:01,168 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:39:01,168 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 06:39:01,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 06:39:01,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-4tdj.c.gcp-upstream.internal','N') ;"
2026-08-10 06:39:01,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:39:02,153 [I] Table x_portal_user does not exist in database ranger
2026-08-10 06:39:02,153 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:39:02,395 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 06:39:02,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:39:02,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:39:02,896 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 06:39:02,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 06:39:03.803 UTC [18527] WARNING: there is no transaction in progress
2026-08-10 06:39:03.818 UTC [18527] WARNING: there is no transaction in progress
2026-08-10 06:39:03.833 UTC [18527] WARNING: there is no transaction in progress
2026-08-10 06:39:03.848 UTC [18527] WARNING: there is no transaction in progress
2026-08-10 06:39:03.863 UTC [18527] WARNING: there is no transaction in progress
2026-08-10 06:39:03.881 UTC [18527] WARNING: there is no transaction in progress
2026-08-10 06:39:03.933 UTC [18527] WARNING: there is no transaction in progress
2026-08-10 06:39:03.940 UTC [18527] WARNING: there is no transaction in progress
2026-08-10 06:39:03.953 UTC [18527] WARNING: there is no transaction in progress
2026-08-10 06:39:03.962 UTC [18527] WARNING: there is no transaction in progress
2026-08-10 06:39:03.972 UTC [18527] WARNING: there is no transaction in progress
2026-08-10 06:39:03.986 UTC [18527] WARNING: there is no transaction in progress
2026-08-10 06:39:03.995 UTC [18527] WARNING: there is no transaction in progress
2026-08-10 06:39:04.001 UTC [18527] WARNING: there is no transaction in progress
2026-08-10 06:39:04.007 UTC [18527] WARNING: there is no transaction in progress
2026-08-10 06:39:04.013 UTC [18527] WARNING: there is no transaction in progress
2026-08-10 06:39:04,580 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 06:39:04,580 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 06:39:04,835 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 06:39:04,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:39:05,083 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 06:39:05,083 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:39:05,322 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 06:39:05,323 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:39:05,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:39:05,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-4tdj.c.gcp-upstream.internal';"
2026-08-10 06:39:06,040 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 06:39:06,040 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 06:39:06,292 [I] DB_PATCHES have already been applied
I20260810 06:39:06.298146 31983 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ranger-admin:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskq68xXZ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 06:39:06.298250 31983 mini_ranger.cc:193] Using host: 127.31.59.212
I20260810 06:39:06.317871 31983 mini_ranger.cc:227] Ranger admin URL: http://127.31.59.212:49579
Aug 10, 2026 6:39:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:39:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:39:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 6:39:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 6:39:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 6:39:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 6:39:07 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 06:39:08.164356 31983 mini_ranger.cc:160] Time spent starting Ranger: real 9.134s user 0.010s sys 0.046s
Aug 10, 2026 6:39:27 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 6:39:27 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 6:39:27 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 6:39:27 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 6:39:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 6:39:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 6:39:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 6:39:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 6:39:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 6:39:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 6:39:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 6:39:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 6:39:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 6:39:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 6:39:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 6:39:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 6:39:27 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 6:39:28 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 06:39:29.338703 31983 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.59.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.59.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.59.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/rangerkms_127.31.59.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.59.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/rangerkms_127.31.59.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.59.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.59.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.59.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/HTTP_127.31.59.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.59.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/HTTP_127.31.59.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 06:39:29 dist-test-slave-4tdj krb5kdc[18178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343969, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 06:39:29.516750 31983 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 06:39:29.540758 31983 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 06:39:30.318563 31983 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 06:39:30.318687 31983 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ranger-kms
I20260810 06:39:30.318820 31983 mini_ranger_kms.cc:227] Using postgres at 127.31.59.212:54591
2026-08-10 06:39:30,789 [I] DB FLAVOR :POSTGRES
2026-08-10 06:39:30,789 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:39:30,789 [I] Checking connection
2026-08-10 06:39:30,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 06:39:31,034 [I] connection success
2026-08-10 06:39:31,035 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 06:39:31,035 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 06:39:31,035 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 06:39:31,272 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 06:39:31,272 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:39:31,272 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 06:39:31,272 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:54591/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 06:39:31,519 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 06:39:32.004980 31983 mini_ranger_kms.cc:310] Created kms service
I20260810 06:39:32.187944 31983 mini_ranger_kms.cc:326] Created kudu user
I20260810 06:39:32.228720 31983 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 06:39:32 dist-test-slave-4tdj krb5kdc[18178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343969, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:39:32.619400 31983 mini_ranger_kms.cc:384] Added ranger policy
I20260810 06:39:32.619777 31983 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ranger-kms:/tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 06:39:32.619845 31983 mini_ranger_kms.cc:240] Using host: 127.31.59.212
I20260810 06:39:32.647281 31983 mini_ranger_kms.cc:276] Ranger KMS PID: 18842
I20260810 06:39:32.647446 31983 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.59.212:37421
Aug 10, 2026 6:39:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:39:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:39:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 6:39:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 6:39:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 6:39:34 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 06:39:34 dist-test-slave-4tdj krb5kdc[18178](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786343974, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.59.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 06:39:35 dist-test-slave-4tdj krb5kdc[18178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343974, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.59.212@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:39:35.295715 31983 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.779s user 0.118s sys 0.380s
I20260810 06:39:35.296054 31983 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 06:39:35.296257 31983 mini_ranger_kms.cc:401] 127.31.59.212:37421/kms/v1/keys
Aug 10 06:39:35 dist-test-slave-4tdj krb5kdc[18178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343974, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.59.212@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
Aug 10 06:39:37 dist-test-slave-4tdj krb5kdc[18178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:39:38.049714 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:33655
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:33655
--ranger_config_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.212:37421/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:38.522744 18938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:38.523144 18938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:38.523296 18938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:38.531316 18938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:38.531428 18938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:38.531512 18938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:38.531591 18938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:38.531658 18938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:38.542734 18938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.59.212:37421/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:33655
--ranger_config_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:33655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:39:38.543989 18938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:38.545317 18938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:38.557961 18950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:38.558246 18947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:38.557964 18952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:38.558616 18938 server_base.cc:1034] running on GCE node
I20260810 06:39:38.559628 18938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:38.560984 18938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:38.562279 18938 hybrid_clock.cc:648] HybridClock initialized: now 1786343978562168 us; error 34 us; skew 500 ppm
I20260810 06:39:38.562824 18938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:39:38 dist-test-slave-4tdj krb5kdc[18178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343978, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:38.570113 18938 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:39:38.571735 18938 webserver.cc:460] Webserver started at http://127.31.59.254:38805/ using document root <none> and password file <none>
I20260810 06:39:38.572358 18938 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:38.572455 18938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:38.572830 18938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:39:38 dist-test-slave-4tdj krb5kdc[18178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343978, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:39:38.634438 18938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "70c8f4a60eb84c2f903e333eae07661b"
format_stamp: "Formatted at 2026-08-10 06:39:38 on dist-test-slave-4tdj"
server_key: "3c6ef14b5400fe43eb0a7172622aef33"
server_key_iv: "524f2c5b25bb4c21f377f9cebd70620c"
server_key_version: "kuduclusterkey@0"
I20260810 06:39:38.635659 18938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "70c8f4a60eb84c2f903e333eae07661b"
format_stamp: "Formatted at 2026-08-10 06:39:38 on dist-test-slave-4tdj"
server_key: "3c6ef14b5400fe43eb0a7172622aef33"
server_key_iv: "524f2c5b25bb4c21f377f9cebd70620c"
server_key_version: "kuduclusterkey@0"
I20260810 06:39:38.643472 18938 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.006s
I20260810 06:39:38.664865 18972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:38.666446 18938 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 06:39:38.666613 18938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "70c8f4a60eb84c2f903e333eae07661b"
format_stamp: "Formatted at 2026-08-10 06:39:38 on dist-test-slave-4tdj"
server_key: "3c6ef14b5400fe43eb0a7172622aef33"
server_key_iv: "524f2c5b25bb4c21f377f9cebd70620c"
server_key_version: "kuduclusterkey@0"
I20260810 06:39:38.666834 18938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:38.703557 18938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:38.708578 18938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:38.708945 18938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:38.735862 18938 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:33655
I20260810 06:39:38.735850 19057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:33655 every 8 connection(s)
I20260810 06:39:38.737495 18938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:38.741925 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 18938
I20260810 06:39:38.742136 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:39:38.742897 19058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:38.742978 31983 external_mini_cluster.cc:1428] Setting key 1644db617e2ad469c1205b584800c519
I20260810 06:39:38.763430 19058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b: Bootstrap starting.
I20260810 06:39:38.768150 19058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b: Neither blocks nor log segments found. Creating new log.
Aug 10 06:39:38 dist-test-slave-4tdj krb5kdc[18178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:39:38.769796 19058 log.cc:824] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:38.774559 19058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b: No bootstrap required, opened a new log
I20260810 06:39:38.779772 19064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:38.745525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33090 (local address 127.31.59.254:33655)
0810 06:39:38.754273 (+ 8748us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:38.754322 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:38.754383 (+ 61us) server_negotiation.cc:407] Connection header received
0810 06:39:38.757768 (+ 3385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:38.757801 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:38.758364 (+ 563us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:38.758932 (+ 568us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:38.760543 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.762788 (+ 2245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.764596 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.765633 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.765846 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:38.771014 (+ 5168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:38.771040 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:38.771063 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:38.771109 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:38.775606 (+ 4497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:38.776256 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:38.776263 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:38.776269 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:38.776408 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:38.776777 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:38.776780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:38.776782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:38.777357 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:38.777775 (+ 418us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:38.778268 (+ 493us) server_negotiation.cc:299] Negotiation successful
0810 06:39:38.778704 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8470,"thread_start_us":8079,"threads_started":1}
I20260810 06:39:38.784559 19058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70c8f4a60eb84c2f903e333eae07661b" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33655 } }
I20260810 06:39:38.784907 19058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:38.785004 19058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70c8f4a60eb84c2f903e333eae07661b, State: Initialized, Role: FOLLOWER
I20260810 06:39:38.785553 19058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70c8f4a60eb84c2f903e333eae07661b" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33655 } }
I20260810 06:39:38.785811 19058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:38.786000 19058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:38.786257 19058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:38.789081 19058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70c8f4a60eb84c2f903e333eae07661b" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33655 } }
I20260810 06:39:38.789512 19058 leader_election.cc:302] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70c8f4a60eb84c2f903e333eae07661b; no voters:
I20260810 06:39:38.790201 19058 leader_election.cc:288] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:39:38.790499 19069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:38.791571 19069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b [term 1 LEADER]: Becoming Leader. State: Replica: 70c8f4a60eb84c2f903e333eae07661b, State: Running, Role: LEADER
I20260810 06:39:38.792183 19069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70c8f4a60eb84c2f903e333eae07661b" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33655 } }
I20260810 06:39:38.792780 19058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:38.796609 19071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70c8f4a60eb84c2f903e333eae07661b. Latest consensus state: current_term: 1 leader_uuid: "70c8f4a60eb84c2f903e333eae07661b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70c8f4a60eb84c2f903e333eae07661b" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33655 } } }
I20260810 06:39:38.797461 19071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:38.796877 19070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70c8f4a60eb84c2f903e333eae07661b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70c8f4a60eb84c2f903e333eae07661b" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33655 } } }
I20260810 06:39:38.797698 19070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:38.806149 19086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:39:38.821276 19086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:38.822535 19058 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 06:39:38.851128 19086 catalog_manager.cc:1347] Generated new cluster ID: f60dd5dad41143f5a59bd182ae4b1a61
I20260810 06:39:38.851306 19086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:39:38.869155 19086 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:39:38.871244 19086 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:38.880345 19086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b: Generated new TSK 0
I20260810 06:39:38.881611 19086 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
I20260810 06:39:39.699250 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.212:37421/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:33655
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:39:39 dist-test-slave-4tdj krb5kdc[18178](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786343979, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 06:39:40.226924 19138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:40.227231 19138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:40.227397 19138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:40.235172 19138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:40.235291 19138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:40.235450 19138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:39:40.246428 19138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.59.212:37421/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:33655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:39:40.247809 19138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:40.249182 19138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:40.266763 19151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:40.268769 19153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:40.277567 19138 server_base.cc:1034] running on GCE node
W20260810 06:39:40.278545 19155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:40.279611 19138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:40.280958 19138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:40.282307 19138 hybrid_clock.cc:648] HybridClock initialized: now 1786343980282182 us; error 57 us; skew 500 ppm
I20260810 06:39:40.282804 19138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:39:40 dist-test-slave-4tdj krb5kdc[18178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343980, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:40.287359 19138 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:39:40.291092 19138 webserver.cc:460] Webserver started at http://127.31.59.193:44053/ using document root <none> and password file <none>
I20260810 06:39:40.292016 19138 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:40.292234 19138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:40.292762 19138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:39:40 dist-test-slave-4tdj krb5kdc[18178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343980, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:39:40.317757 19138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "33fffb511d3c46fc8cdef4078defb3d0"
format_stamp: "Formatted at 2026-08-10 06:39:40 on dist-test-slave-4tdj"
server_key: "39b6fd42fedb609bc24227821bd2ec77"
server_key_iv: "4ebe7efa2e37cc01967b359cf7e724da"
server_key_version: "kuduclusterkey@0"
I20260810 06:39:40.319325 19138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "33fffb511d3c46fc8cdef4078defb3d0"
format_stamp: "Formatted at 2026-08-10 06:39:40 on dist-test-slave-4tdj"
server_key: "39b6fd42fedb609bc24227821bd2ec77"
server_key_iv: "4ebe7efa2e37cc01967b359cf7e724da"
server_key_version: "kuduclusterkey@0"
I20260810 06:39:40.327987 19138 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.005s
I20260810 06:39:40.347234 19169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:40.349287 19138 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260810 06:39:40.349532 19138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "33fffb511d3c46fc8cdef4078defb3d0"
format_stamp: "Formatted at 2026-08-10 06:39:40 on dist-test-slave-4tdj"
server_key: "39b6fd42fedb609bc24227821bd2ec77"
server_key_iv: "4ebe7efa2e37cc01967b359cf7e724da"
server_key_version: "kuduclusterkey@0"
I20260810 06:39:40.349817 19138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:40.372896 19138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:40.386376 19138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:40.386729 19138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:40.388204 19138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:40.403365 19138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:40.403606 19138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 06:39:40.403949 19138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:40.404110 19138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:39:40 dist-test-slave-4tdj krb5kdc[18178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343979, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:39:40.727583 19138 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:46511
I20260810 06:39:40.727722 19346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:46511 every 8 connection(s)
I20260810 06:39:40.729744 19138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:39:40.739519 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 19138
I20260810 06:39:40.739750 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:39:40.740878 31983 external_mini_cluster.cc:1428] Setting key 139cd768d4f14ab1e8680da831f8c65d
Aug 10 06:39:40 dist-test-slave-4tdj krb5kdc[18178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343980, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:39:40.768679 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.734195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:33655 (local address 127.31.59.193:51451)
0810 06:39:40.744415 (+ 10220us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:40.744463 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:40.746791 (+ 2328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:40.747861 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:40.747875 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:40.748508 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:40.749367 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.749393 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.750929 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.750934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.753248 (+ 2314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.753301 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.754552 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.754559 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.754836 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.755653 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:39:40.755683 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:40.758817 (+ 3134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:40.763460 (+ 4643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:40.763469 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:40.763487 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:40.764068 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:40.764576 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:40.764580 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:40.764583 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:40.764952 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:40.765699 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:40.765707 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:40.766169 (+ 462us) client_negotiation.cc:769] Sending connection context
0810 06:39:40.766492 (+ 323us) client_negotiation.cc:240] Negotiation successful
0810 06:39:40.767107 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9955,"thread_start_us":9718,"threads_started":1}
I20260810 06:39:40.769766 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.733419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:51451 (local address 127.31.59.254:33655)
0810 06:39:40.743867 (+ 10448us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.743872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.744586 (+ 714us) server_negotiation.cc:407] Connection header received
0810 06:39:40.747217 (+ 2631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.747222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.747296 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.747393 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:40.749593 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.750705 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.753455 (+ 2750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.754093 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.754188 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.759133 (+ 4945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:40.759140 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:40.759144 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:40.759186 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:40.763292 (+ 4106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:40.764226 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:40.764232 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:40.764235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:40.764442 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:40.765091 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:40.765096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:40.765099 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:40.765493 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:40.765619 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.769179 (+ 3560us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.769442 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10332,"thread_start_us":12888,"threads_started":1}
I20260810 06:39:40.771801 19347 heartbeater.cc:347] Connected to a master server at 127.31.59.254:33655
I20260810 06:39:40.772171 19347 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:40.773154 19347 heartbeater.cc:511] Master 127.31.59.254:33655 requested a full tablet report, sending...
I20260810 06:39:40.775695 19000 ts_manager.cc:194] Registered new tserver with Master: 33fffb511d3c46fc8cdef4078defb3d0 (127.31.59.193:46511)
I20260810 06:39:40.777778 19000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:51451
WARNING: no policy specified for kudu/127.31.59.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
I20260810 06:39:40.811432 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.212:37421/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:33655
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:39:40 dist-test-slave-4tdj krb5kdc[18178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343979, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
W20260810 06:39:41.418126 19357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:41.418437 19357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:41.418582 19357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:41.426256 19357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:41.426395 19357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:41.426559 19357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:39:41.437851 19357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.59.212:37421/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:33655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:39:41.439131 19357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:41.440691 19357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:41.456348 19380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:41.456929 19385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:41.457434 19381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:41.458361 19357 server_base.cc:1034] running on GCE node
I20260810 06:39:41.458947 19357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:41.460009 19357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:41.461325 19357 hybrid_clock.cc:648] HybridClock initialized: now 1786343981461183 us; error 95 us; skew 500 ppm
I20260810 06:39:41.461766 19357 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:39:41 dist-test-slave-4tdj krb5kdc[18178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343981, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:41.465191 19357 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:39:41.467229 19357 webserver.cc:460] Webserver started at http://127.31.59.194:39353/ using document root <none> and password file <none>
I20260810 06:39:41.468055 19357 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:41.468223 19357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:41.468667 19357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:39:41 dist-test-slave-4tdj krb5kdc[18178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343981, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:39:41.487572 19357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "93dbf6fb30d94465bed126c36502af73"
format_stamp: "Formatted at 2026-08-10 06:39:41 on dist-test-slave-4tdj"
server_key: "19747b0196c36672ea1b06e684b7a022"
server_key_iv: "95c7d0c04365efe44656f47545a377f4"
server_key_version: "kuduclusterkey@0"
I20260810 06:39:41.488658 19357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "93dbf6fb30d94465bed126c36502af73"
format_stamp: "Formatted at 2026-08-10 06:39:41 on dist-test-slave-4tdj"
server_key: "19747b0196c36672ea1b06e684b7a022"
server_key_iv: "95c7d0c04365efe44656f47545a377f4"
server_key_version: "kuduclusterkey@0"
I20260810 06:39:41.497370 19357 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.005s
I20260810 06:39:41.515702 19395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:41.517668 19357 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260810 06:39:41.517866 19357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "93dbf6fb30d94465bed126c36502af73"
format_stamp: "Formatted at 2026-08-10 06:39:41 on dist-test-slave-4tdj"
server_key: "19747b0196c36672ea1b06e684b7a022"
server_key_iv: "95c7d0c04365efe44656f47545a377f4"
server_key_version: "kuduclusterkey@0"
I20260810 06:39:41.518147 19357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:41.545816 19357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:41.551280 19357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:41.551631 19357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:41.552712 19357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:41.554723 19357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:41.554802 19357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:41.554919 19357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:41.554970 19357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:41.615450 19357 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:40505
I20260810 06:39:41.615445 19572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:40505 every 8 connection(s)
I20260810 06:39:41.617112 19357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:39:41.625429 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 19357
I20260810 06:39:41.625626 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:39:41.626190 31983 external_mini_cluster.cc:1428] Setting key 335e512bbce94c58c0312cccae9d8a08
Aug 10 06:39:41 dist-test-slave-4tdj krb5kdc[18178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343981, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:39:41.641978 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:41.620233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:43485 (local address 127.31.59.254:33655)
0810 06:39:41.620781 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:41.620786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:41.621969 (+ 1183us) server_negotiation.cc:407] Connection header received
0810 06:39:41.624512 (+ 2543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:41.624519 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:41.624608 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:41.624727 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:41.626933 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:41.628035 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:41.629707 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:41.630358 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:41.630471 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:41.634630 (+ 4159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:41.634636 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:41.634639 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:41.634671 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:41.638463 (+ 3792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:41.639170 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:41.639175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:41.639177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:41.639293 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:41.639795 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:41.639800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:41.639803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:41.640241 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:41.640384 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:41.641295 (+ 911us) server_negotiation.cc:299] Negotiation successful
0810 06:39:41.641527 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":375,"thread_start_us":203,"threads_started":1}
I20260810 06:39:41.643165 19578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:41.620992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:33655 (local address 127.31.59.194:43485)
0810 06:39:41.621809 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:41.621853 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:41.624234 (+ 2381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:41.625132 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:41.625140 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:41.625896 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:41.626720 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:41.626742 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:41.628212 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:41.628218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:41.629561 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:41.629581 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:41.630583 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:41.630591 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:41.630896 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:41.631652 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:39:41.631702 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:41.634351 (+ 2649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:41.638605 (+ 4254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:41.638613 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:41.638630 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:41.639046 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:41.639397 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:41.639402 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:41.639406 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:41.639657 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:41.640399 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:41.640407 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:41.640850 (+ 443us) client_negotiation.cc:769] Sending connection context
0810 06:39:41.641178 (+ 328us) client_negotiation.cc:240] Negotiation successful
0810 06:39:41.641743 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":357,"threads_started":1}
I20260810 06:39:41.644773 19573 heartbeater.cc:347] Connected to a master server at 127.31.59.254:33655
I20260810 06:39:41.645183 19573 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:41.646611 19573 heartbeater.cc:511] Master 127.31.59.254:33655 requested a full tablet report, sending...
I20260810 06:39:41.649442 19000 ts_manager.cc:194] Registered new tserver with Master: 93dbf6fb30d94465bed126c36502af73 (127.31.59.194:40505)
I20260810 06:39:41.650646 19000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:43485
WARNING: no policy specified for kudu/127.31.59.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
I20260810 06:39:41.689100 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.212:37421/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:33655
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:39:41.781572 19347 heartbeater.cc:503] Master 127.31.59.254:33655 was elected leader, sending a full tablet report...
W20260810 06:39:42.136356 19581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:42.136657 19581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:42.136798 19581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:42.144789 19581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:42.144950 19581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:42.145165 19581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:39:42.156085 19581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.59.212:37421/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:33655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:39:42.157354 19581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:42.158689 19581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:42.172597 19594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:42.172739 19591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:42.173167 19587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:42.173661 19581 server_base.cc:1034] running on GCE node
I20260810 06:39:42.174234 19581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:42.175127 19581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:42.176339 19581 hybrid_clock.cc:648] HybridClock initialized: now 1786343982176285 us; error 50 us; skew 500 ppm
I20260810 06:39:42.176688 19581 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:39:42 dist-test-slave-4tdj krb5kdc[18178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343982, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:42.180109 19581 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:39:42.181382 19581 webserver.cc:460] Webserver started at http://127.31.59.195:32971/ using document root <none> and password file <none>
I20260810 06:39:42.181964 19581 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:42.182065 19581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:42.182348 19581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:39:42 dist-test-slave-4tdj krb5kdc[18178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343982, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:39:42.196209 19581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "74c0a2ad395e4019b328d81353f27eda"
format_stamp: "Formatted at 2026-08-10 06:39:42 on dist-test-slave-4tdj"
server_key: "0398a1a9179e0ce073ec3a22a847b228"
server_key_iv: "2368cbe104a15048ebb5f9d07b6c7fb1"
server_key_version: "kuduclusterkey@0"
I20260810 06:39:42.197002 19581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "74c0a2ad395e4019b328d81353f27eda"
format_stamp: "Formatted at 2026-08-10 06:39:42 on dist-test-slave-4tdj"
server_key: "0398a1a9179e0ce073ec3a22a847b228"
server_key_iv: "2368cbe104a15048ebb5f9d07b6c7fb1"
server_key_version: "kuduclusterkey@0"
I20260810 06:39:42.202553 19581 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 06:39:42.216782 19603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:42.218252 19581 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260810 06:39:42.218453 19581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "74c0a2ad395e4019b328d81353f27eda"
format_stamp: "Formatted at 2026-08-10 06:39:42 on dist-test-slave-4tdj"
server_key: "0398a1a9179e0ce073ec3a22a847b228"
server_key_iv: "2368cbe104a15048ebb5f9d07b6c7fb1"
server_key_version: "kuduclusterkey@0"
I20260810 06:39:42.218675 19581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:42.242133 19581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:42.246392 19581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:42.246693 19581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:42.247653 19581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:42.249154 19581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:42.249233 19581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:42.249320 19581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:42.249364 19581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:42.290935 19581 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:33579
I20260810 06:39:42.290977 19790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:33579 every 8 connection(s)
I20260810 06:39:42.292265 19581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:39:42.297461 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 19581
I20260810 06:39:42.297647 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:39:42.298220 31983 external_mini_cluster.cc:1428] Setting key 29b28b833db426ca59c61008826d9802
Aug 10 06:39:42 dist-test-slave-4tdj krb5kdc[18178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343982, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:39:42.314124 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.295073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:49611 (local address 127.31.59.254:33655)
0810 06:39:42.295580 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.295584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.296601 (+ 1017us) server_negotiation.cc:407] Connection header received
0810 06:39:42.299222 (+ 2621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.299228 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.299312 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.299429 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:42.301241 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.301961 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.303812 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.304264 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.304372 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.307636 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:42.307641 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:42.307645 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:42.307683 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:42.310525 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.311262 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.311266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.311269 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.311419 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.312070 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.312074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.312076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.312394 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:42.312552 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.313317 (+ 765us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.313546 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":238,"threads_started":1}
I20260810 06:39:42.314983 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.295574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:33655 (local address 127.31.59.195:49611)
0810 06:39:42.296437 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:42.296474 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:42.298931 (+ 2457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:42.299674 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:42.299683 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:42.300368 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:42.301073 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.301101 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.302174 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.302178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.303669 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.303681 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.304491 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.304499 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.304707 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.305298 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:39:42.305324 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:42.307390 (+ 2066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:42.310657 (+ 3267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:42.310667 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:42.310683 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:42.311124 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:42.311533 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:42.311539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:42.311543 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:42.311762 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:42.312530 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:42.312538 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:42.312910 (+ 372us) client_negotiation.cc:769] Sending connection context
0810 06:39:42.313203 (+ 293us) client_negotiation.cc:240] Negotiation successful
0810 06:39:42.313732 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":411,"threads_started":1}
I20260810 06:39:42.316222 19791 heartbeater.cc:347] Connected to a master server at 127.31.59.254:33655
I20260810 06:39:42.316516 19791 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:42.317265 19791 heartbeater.cc:511] Master 127.31.59.254:33655 requested a full tablet report, sending...
I20260810 06:39:42.318827 19000 ts_manager.cc:194] Registered new tserver with Master: 74c0a2ad395e4019b328d81353f27eda (127.31.59.195:33579)
I20260810 06:39:42.319795 19000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:49611
I20260810 06:39:42.325664 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:39:42.342866 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.330534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33092 (local address 127.31.59.254:33655)
0810 06:39:42.330755 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.330760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.331041 (+ 281us) server_negotiation.cc:407] Connection header received
0810 06:39:42.331229 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.331232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.331302 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.331373 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:42.333075 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.333769 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.335463 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.335877 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.335967 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.338127 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:42.338131 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:42.338135 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:42.338163 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:42.340415 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.341171 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.341174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.341176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.341280 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.341622 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.341625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.341627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.341880 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:42.342053 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.342425 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.342608 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:39:42.346525 19000 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:39:42.354453 19000 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:39:42.387240 19814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.367307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49574 (local address 127.31.59.195:33579)
0810 06:39:42.374334 (+ 7027us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.374343 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.377366 (+ 3023us) server_negotiation.cc:407] Connection header received
0810 06:39:42.377477 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.377486 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.377794 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.377970 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:42.379127 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.380731 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.384376 (+ 3645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.385925 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.386048 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.386329 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.386727 (+ 398us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.386893 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6827,"thread_start_us":6621,"threads_started":1}
I20260810 06:39:42.387280 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.367337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:33579 (local address 127.0.0.1:49574)
0810 06:39:42.376925 (+ 9588us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:42.376965 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:42.377167 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:42.378323 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:42.378332 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:42.378373 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:42.378903 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.378917 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.380887 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.380891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.384179 (+ 3288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.384193 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.386190 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.386197 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.386448 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.386541 (+ 93us) client_negotiation.cc:769] Sending connection context
0810 06:39:42.386646 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 06:39:42.386924 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9330,"mutex_wait_us":22,"thread_start_us":9159,"threads_started":1}
I20260810 06:39:42.390892 19687 tablet_service.cc:1467] Processing CreateTablet for tablet e6605f26b2284a9394fe807721b64193 (DEFAULT_TABLE table=test-table [id=1646e86e972b4df2b17a6d7b37620b51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:42.392519 19687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6605f26b2284a9394fe807721b64193. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:42.393846 19822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.370175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46562 (local address 127.31.59.193:46511)
0810 06:39:42.374589 (+ 4414us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.374597 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.385006 (+ 10409us) server_negotiation.cc:407] Connection header received
0810 06:39:42.385381 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.385389 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.385606 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.385804 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:42.387025 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.388638 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.391445 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.392475 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.392609 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.392926 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.393265 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.393439 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4230,"thread_start_us":4045,"threads_started":1}
I20260810 06:39:42.393940 19821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.367544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:40505 (local address 127.0.0.1:48080)
0810 06:39:42.383845 (+ 16301us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:42.383875 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:42.384016 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:42.385719 (+ 1703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:42.385724 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:42.385750 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:42.386623 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.386638 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.388649 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.388655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.391025 (+ 2370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.391041 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.393182 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.393189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.393363 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.393395 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:39:42.393463 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 06:39:42.393582 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16129,"mutex_wait_us":98,"thread_start_us":17013,"threads_started":1}
I20260810 06:39:42.394752 19818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.369520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:46511 (local address 127.0.0.1:46562)
0810 06:39:42.384881 (+ 15361us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:42.384900 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:42.385157 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:42.386060 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:42.386065 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:42.386106 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:42.386813 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.386825 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.388819 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.388825 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.391290 (+ 2465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.391300 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.392761 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.392769 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.393013 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.393049 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:39:42.394355 (+ 1306us) client_negotiation.cc:240] Negotiation successful
0810 06:39:42.394469 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15218,"mutex_wait_us":85,"thread_start_us":14619,"threads_started":1}
I20260810 06:39:42.394812 19820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.367628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48080 (local address 127.31.59.194:40505)
0810 06:39:42.385153 (+ 17525us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.385162 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.385189 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:39:42.385310 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.385319 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.385542 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.385719 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:42.386882 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.388470 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.391196 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.392552 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.392694 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.394246 (+ 1552us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.394343 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.394508 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17349,"thread_start_us":17224,"threads_started":1}
I20260810 06:39:42.397796 19468 tablet_service.cc:1467] Processing CreateTablet for tablet e6605f26b2284a9394fe807721b64193 (DEFAULT_TABLE table=test-table [id=1646e86e972b4df2b17a6d7b37620b51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:42.398670 19244 tablet_service.cc:1467] Processing CreateTablet for tablet e6605f26b2284a9394fe807721b64193 (DEFAULT_TABLE table=test-table [id=1646e86e972b4df2b17a6d7b37620b51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:42.399422 19468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6605f26b2284a9394fe807721b64193. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:42.400223 19244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6605f26b2284a9394fe807721b64193. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:42.409114 19826 tablet_bootstrap.cc:492] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda: Bootstrap starting.
I20260810 06:39:42.413516 19828 tablet_bootstrap.cc:492] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73: Bootstrap starting.
I20260810 06:39:42.414520 19826 tablet_bootstrap.cc:654] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:42.415192 19829 tablet_bootstrap.cc:492] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0: Bootstrap starting.
I20260810 06:39:42.417572 19826 log.cc:824] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:42.418604 19828 tablet_bootstrap.cc:654] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:42.419776 19829 tablet_bootstrap.cc:654] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:42.420594 19828 log.cc:824] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:42.421919 19829 log.cc:824] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:42.422242 19826 tablet_bootstrap.cc:492] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda: No bootstrap required, opened a new log
I20260810 06:39:42.422643 19826 ts_tablet_manager.cc:1397] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.006s
I20260810 06:39:42.424933 19828 tablet_bootstrap.cc:492] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73: No bootstrap required, opened a new log
I20260810 06:39:42.425347 19828 ts_tablet_manager.cc:1397] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260810 06:39:42.425976 19829 tablet_bootstrap.cc:492] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0: No bootstrap required, opened a new log
I20260810 06:39:42.426357 19829 ts_tablet_manager.cc:1397] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260810 06:39:42.434979 19826 raft_consensus.cc:348] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c0a2ad395e4019b328d81353f27eda" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 33579 } } peers { permanent_uuid: "93dbf6fb30d94465bed126c36502af73" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40505 } } peers { permanent_uuid: "33fffb511d3c46fc8cdef4078defb3d0" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46511 } }
I20260810 06:39:42.435236 19828 raft_consensus.cc:348] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c0a2ad395e4019b328d81353f27eda" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 33579 } } peers { permanent_uuid: "93dbf6fb30d94465bed126c36502af73" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40505 } } peers { permanent_uuid: "33fffb511d3c46fc8cdef4078defb3d0" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46511 } }
I20260810 06:39:42.435563 19826 raft_consensus.cc:374] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:42.435600 19828 raft_consensus.cc:374] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:42.435660 19826 raft_consensus.cc:729] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74c0a2ad395e4019b328d81353f27eda, State: Initialized, Role: FOLLOWER
I20260810 06:39:42.435702 19828 raft_consensus.cc:729] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93dbf6fb30d94465bed126c36502af73, State: Initialized, Role: FOLLOWER
I20260810 06:39:42.436305 19828 consensus_queue.cc:260] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c0a2ad395e4019b328d81353f27eda" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 33579 } } peers { permanent_uuid: "93dbf6fb30d94465bed126c36502af73" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40505 } } peers { permanent_uuid: "33fffb511d3c46fc8cdef4078defb3d0" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46511 } }
I20260810 06:39:42.436305 19826 consensus_queue.cc:260] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c0a2ad395e4019b328d81353f27eda" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 33579 } } peers { permanent_uuid: "93dbf6fb30d94465bed126c36502af73" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40505 } } peers { permanent_uuid: "33fffb511d3c46fc8cdef4078defb3d0" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46511 } }
I20260810 06:39:42.438236 19573 heartbeater.cc:503] Master 127.31.59.254:33655 was elected leader, sending a full tablet report...
I20260810 06:39:42.438335 19828 ts_tablet_manager.cc:1428] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73: Time spent starting tablet: real 0.013s user 0.004s sys 0.009s
I20260810 06:39:42.438697 19826 ts_tablet_manager.cc:1428] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda: Time spent starting tablet: real 0.016s user 0.010s sys 0.006s
I20260810 06:39:42.438652 19829 raft_consensus.cc:348] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c0a2ad395e4019b328d81353f27eda" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 33579 } } peers { permanent_uuid: "93dbf6fb30d94465bed126c36502af73" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40505 } } peers { permanent_uuid: "33fffb511d3c46fc8cdef4078defb3d0" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46511 } }
I20260810 06:39:42.439177 19829 raft_consensus.cc:374] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:42.439333 19829 raft_consensus.cc:729] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33fffb511d3c46fc8cdef4078defb3d0, State: Initialized, Role: FOLLOWER
I20260810 06:39:42.439370 19791 heartbeater.cc:503] Master 127.31.59.254:33655 was elected leader, sending a full tablet report...
I20260810 06:39:42.440168 19829 consensus_queue.cc:260] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c0a2ad395e4019b328d81353f27eda" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 33579 } } peers { permanent_uuid: "93dbf6fb30d94465bed126c36502af73" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40505 } } peers { permanent_uuid: "33fffb511d3c46fc8cdef4078defb3d0" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46511 } }
I20260810 06:39:42.442198 19829 ts_tablet_manager.cc:1428] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0: Time spent starting tablet: real 0.016s user 0.013s sys 0.002s
W20260810 06:39:42.494454 19349 tablet.cc:2369] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:39:42.544456 19793 tablet.cc:2369] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:39:42.572613 19838 raft_consensus.cc:482] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:39:42.572980 19838 raft_consensus.cc:504] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c0a2ad395e4019b328d81353f27eda" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 33579 } } peers { permanent_uuid: "93dbf6fb30d94465bed126c36502af73" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40505 } } peers { permanent_uuid: "33fffb511d3c46fc8cdef4078defb3d0" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46511 } }
I20260810 06:39:42.574774 19838 leader_election.cc:288] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74c0a2ad395e4019b328d81353f27eda (127.31.59.195:33579), 93dbf6fb30d94465bed126c36502af73 (127.31.59.194:40505)
I20260810 06:39:42.581653 19814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.575175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:60597 (local address 127.31.59.195:33579)
0810 06:39:42.575453 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.575458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.576427 (+ 969us) server_negotiation.cc:407] Connection header received
0810 06:39:42.576668 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.576673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.576735 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.576809 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:42.577386 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.578084 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.579809 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.580784 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.580931 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.581006 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.581289 (+ 283us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.581406 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260810 06:39:42.581880 19839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.575196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:33579 (local address 127.31.59.193:60597)
0810 06:39:42.576319 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:42.576336 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:42.576476 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:42.576841 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:42.576845 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:42.576871 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:42.577253 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.577266 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.578206 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.578212 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.579661 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.579673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.580979 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.580985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.581140 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.581167 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 06:39:42.581247 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:39:42.581366 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1017,"thread_start_us":622,"threads_started":1}
I20260810 06:39:42.582501 19820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.575313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:54687 (local address 127.31.59.194:40505)
0810 06:39:42.575596 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.575602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.576892 (+ 1290us) server_negotiation.cc:407] Connection header received
0810 06:39:42.577063 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.577077 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.577153 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.577238 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:42.577872 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.578658 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.580305 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.581367 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.581501 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.581570 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.582085 (+ 515us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.582212 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:39:42.583113 19722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6605f26b2284a9394fe807721b64193" candidate_uuid: "33fffb511d3c46fc8cdef4078defb3d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74c0a2ad395e4019b328d81353f27eda" is_pre_election: true
I20260810 06:39:42.583592 19722 raft_consensus.cc:2454] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33fffb511d3c46fc8cdef4078defb3d0 in term 0.
I20260810 06:39:42.584000 19842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.575435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:40505 (local address 127.31.59.193:54687)
0810 06:39:42.576793 (+ 1358us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:42.576807 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:42.576936 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:42.577288 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:42.577292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:42.577314 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:42.577710 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.577720 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.578792 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.578797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.580156 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.580168 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.581665 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.581671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.581873 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.581904 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:39:42.582049 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 06:39:42.582164 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1258,"thread_start_us":1050,"threads_started":1}
I20260810 06:39:42.584270 19172 leader_election.cc:302] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33fffb511d3c46fc8cdef4078defb3d0, 74c0a2ad395e4019b328d81353f27eda; no voters:
I20260810 06:39:42.584627 19838 raft_consensus.cc:2781] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:39:42.584743 19838 raft_consensus.cc:482] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:39:42.584838 19838 raft_consensus.cc:3037] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:42.584975 19510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6605f26b2284a9394fe807721b64193" candidate_uuid: "33fffb511d3c46fc8cdef4078defb3d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93dbf6fb30d94465bed126c36502af73" is_pre_election: true
I20260810 06:39:42.585417 19510 raft_consensus.cc:2454] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33fffb511d3c46fc8cdef4078defb3d0 in term 0.
I20260810 06:39:42.587157 19838 raft_consensus.cc:504] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c0a2ad395e4019b328d81353f27eda" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 33579 } } peers { permanent_uuid: "93dbf6fb30d94465bed126c36502af73" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40505 } } peers { permanent_uuid: "33fffb511d3c46fc8cdef4078defb3d0" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46511 } }
I20260810 06:39:42.587951 19838 leader_election.cc:288] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [CANDIDATE]: Term 1 election: Requested vote from peers 74c0a2ad395e4019b328d81353f27eda (127.31.59.195:33579), 93dbf6fb30d94465bed126c36502af73 (127.31.59.194:40505)
I20260810 06:39:42.588228 19722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6605f26b2284a9394fe807721b64193" candidate_uuid: "33fffb511d3c46fc8cdef4078defb3d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74c0a2ad395e4019b328d81353f27eda"
I20260810 06:39:42.588402 19722 raft_consensus.cc:3037] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:42.588701 19510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6605f26b2284a9394fe807721b64193" candidate_uuid: "33fffb511d3c46fc8cdef4078defb3d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93dbf6fb30d94465bed126c36502af73"
I20260810 06:39:42.588981 19510 raft_consensus.cc:3037] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:42.591468 19722 raft_consensus.cc:2454] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33fffb511d3c46fc8cdef4078defb3d0 in term 1.
I20260810 06:39:42.591847 19172 leader_election.cc:302] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33fffb511d3c46fc8cdef4078defb3d0, 74c0a2ad395e4019b328d81353f27eda; no voters:
I20260810 06:39:42.592121 19838 raft_consensus.cc:2781] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:42.592195 19510 raft_consensus.cc:2454] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33fffb511d3c46fc8cdef4078defb3d0 in term 1.
I20260810 06:39:42.592691 19838 raft_consensus.cc:686] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [term 1 LEADER]: Becoming Leader. State: Replica: 33fffb511d3c46fc8cdef4078defb3d0, State: Running, Role: LEADER
I20260810 06:39:42.593292 19838 consensus_queue.cc:237] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c0a2ad395e4019b328d81353f27eda" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 33579 } } peers { permanent_uuid: "93dbf6fb30d94465bed126c36502af73" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40505 } } peers { permanent_uuid: "33fffb511d3c46fc8cdef4078defb3d0" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46511 } }
I20260810 06:39:42.597658 18999 catalog_manager.cc:5495] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33fffb511d3c46fc8cdef4078defb3d0 (127.31.59.193). New cstate: current_term: 1 leader_uuid: "33fffb511d3c46fc8cdef4078defb3d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c0a2ad395e4019b328d81353f27eda" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 33579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93dbf6fb30d94465bed126c36502af73" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33fffb511d3c46fc8cdef4078defb3d0" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 46511 } health_report { overall_health: HEALTHY } } }
W20260810 06:39:42.620105 19575 tablet.cc:2369] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:39:42.626394 19822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.619233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46578 (local address 127.31.59.193:46511)
0810 06:39:42.619452 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.619459 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.619497 (+ 38us) server_negotiation.cc:407] Connection header received
0810 06:39:42.619695 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.619699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.619778 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.619945 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:39:42.620544 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.621639 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.623901 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.624629 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.624779 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.625354 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:39:42.625846 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:39:42.625953 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.626079 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.626143 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 06:39:42.634186 19510 raft_consensus.cc:1261] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73 [term 1 FOLLOWER]: Refusing update from remote peer 33fffb511d3c46fc8cdef4078defb3d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:39:42.634253 19722 raft_consensus.cc:1261] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda [term 1 FOLLOWER]: Refusing update from remote peer 33fffb511d3c46fc8cdef4078defb3d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:39:42.635098 19843 consensus_queue.cc:1035] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a2ad395e4019b328d81353f27eda" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 33579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:39:42.635437 19838 consensus_queue.cc:1035] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93dbf6fb30d94465bed126c36502af73" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:39:42.648387 19847 mvcc.cc:205] Tried to move back new op lower bound from 7316864952860196864 to 7316864952705982464. Current Snapshot: MvccSnapshot[applied={T|T < 7316864952860196864}]
I20260810 06:39:42.663601 18999 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:39:42.663875 18999 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:39:42.667936 18999 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70c8f4a60eb84c2f903e333eae07661b: Sending DeleteTablet for 3 replicas of tablet e6605f26b2284a9394fe807721b64193
I20260810 06:39:42.668975 19687 tablet_service.cc:1514] Processing DeleteTablet for tablet e6605f26b2284a9394fe807721b64193 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:42 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:49574
I20260810 06:39:42.669020 19468 tablet_service.cc:1514] Processing DeleteTablet for tablet e6605f26b2284a9394fe807721b64193 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:42 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:48080
I20260810 06:39:42.669299 19244 tablet_service.cc:1514] Processing DeleteTablet for tablet e6605f26b2284a9394fe807721b64193 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:42 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:46562
I20260810 06:39:42.669947 19866 tablet_replica.cc:330] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73: stopping tablet replica
I20260810 06:39:42.670373 19866 raft_consensus.cc:2229] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:39:42.670964 19866 raft_consensus.cc:2258] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:39:42.671777 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 19138
I20260810 06:39:42.672048 19867 tablet_replica.cc:330] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda: stopping tablet replica
I20260810 06:39:42.672417 19867 raft_consensus.cc:2229] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:39:42.672863 19867 raft_consensus.cc:2258] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:39:42.676116 19866 ts_tablet_manager.cc:1905] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:39:42.683293 19866 ts_tablet_manager.cc:1918] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:39:42.683466 19866 log.cc:1196] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/e6605f26b2284a9394fe807721b64193
I20260810 06:39:42.683983 19866 ts_tablet_manager.cc:1939] T e6605f26b2284a9394fe807721b64193 P 93dbf6fb30d94465bed126c36502af73: Deleting consensus metadata
I20260810 06:39:42.684846 19868 tablet_replica.cc:330] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0: stopping tablet replica
I20260810 06:39:42.685425 19868 raft_consensus.cc:2229] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:39:42.685875 18979 catalog_manager.cc:4841] TS 93dbf6fb30d94465bed126c36502af73 (127.31.59.194:40505): tablet e6605f26b2284a9394fe807721b64193 (table test-table [id=1646e86e972b4df2b17a6d7b37620b51]) successfully deleted
I20260810 06:39:42.686170 19868 raft_consensus.cc:2258] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:39:42.686604 19867 ts_tablet_manager.cc:1905] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:39:42.688012 19868 ts_tablet_manager.cc:1905] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:39:42.694828 19867 ts_tablet_manager.cc:1918] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:39:42.694994 19867 log.cc:1196] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/e6605f26b2284a9394fe807721b64193
I20260810 06:39:42.695513 19867 ts_tablet_manager.cc:1939] T e6605f26b2284a9394fe807721b64193 P 74c0a2ad395e4019b328d81353f27eda: Deleting consensus metadata
I20260810 06:39:42.695747 19868 ts_tablet_manager.cc:1918] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:39:42.695976 19868 log.cc:1196] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/e6605f26b2284a9394fe807721b64193
Aug 10 06:39:42 dist-test-slave-4tdj krb5kdc[18178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:39:42.696733 19868 ts_tablet_manager.cc:1939] T e6605f26b2284a9394fe807721b64193 P 33fffb511d3c46fc8cdef4078defb3d0: Deleting consensus metadata
I20260810 06:39:42.697490 18974 catalog_manager.cc:4841] TS 74c0a2ad395e4019b328d81353f27eda (127.31.59.195:33579): tablet e6605f26b2284a9394fe807721b64193 (table test-table [id=1646e86e972b4df2b17a6d7b37620b51]) successfully deleted
I20260810 06:39:42.698385 18975 catalog_manager.cc:4841] TS 33fffb511d3c46fc8cdef4078defb3d0 (127.31.59.193:46511): tablet e6605f26b2284a9394fe807721b64193 (table test-table [id=1646e86e972b4df2b17a6d7b37620b51]) successfully deleted
I20260810 06:39:42.704523 19822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.673479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46584 (local address 127.31.59.193:46511)
0810 06:39:42.673999 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.674007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.684738 (+ 10731us) server_negotiation.cc:407] Connection header received
0810 06:39:42.685229 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.685235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.685326 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.685409 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:42.688082 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.689234 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.691168 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.691883 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.692000 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.697953 (+ 5953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:42.697965 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:42.697986 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:42.698028 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:42.701356 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.702228 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.702235 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.702237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.702430 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.702933 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.702937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.702940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.703303 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:42.703529 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.703897 (+ 368us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.704075 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377}
I20260810 06:39:42.705322 19341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:39:42.801932 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 19138
I20260810 06:39:42.826787 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 19357
Aug 10 06:39:42 dist-test-slave-4tdj krb5kdc[18178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.194@KRBTEST.COM
I20260810 06:39:42.841638 19820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.827527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48088 (local address 127.31.59.194:40505)
0810 06:39:42.827744 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.827750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.827771 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:39:42.827902 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.827906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.827981 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.828075 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:42.829297 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.830028 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.831532 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.831930 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.832048 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.836050 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:42.836064 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:42.836078 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:42.836109 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:42.838773 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.839426 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.839433 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.839435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.839567 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.839945 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.839949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.839951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.840413 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:42.840684 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.841020 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.841259 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260810 06:39:42.842423 19561 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:39:42.930344 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 19357
I20260810 06:39:42.952286 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 19581
Aug 10 06:39:42 dist-test-slave-4tdj krb5kdc[18178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.195@KRBTEST.COM
I20260810 06:39:42.966264 19814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.953011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49584 (local address 127.31.59.195:33579)
0810 06:39:42.953167 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.953171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.953189 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:39:42.953329 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.953332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.953397 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.953480 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:42.954800 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.955494 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.956704 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.957077 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.957166 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.961318 (+ 4152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:42.961332 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:42.961349 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:42.961387 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:42.963932 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.964622 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.964629 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.964631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.964762 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.965034 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.965036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.965038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.965375 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:42.965610 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.965856 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.966010 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:39:42.967015 19786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:39:43.060541 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 19581
I20260810 06:39:43.080551 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 18938
I20260810 06:39:43.081714 19045 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:39:43.192020 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 18938
I20260810 06:39:43.219097 31983 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 06:39:43.563899 31983 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 06:39:43.564189 31983 mini_ranger.cc:66] Stopping Ranger...
I20260810 06:39:43.908633 31983 mini_ranger.cc:68] Stopped Ranger
2026-08-10 06:39:43.908 UTC [18229] LOG: received smart shutdown request
2026-08-10 06:39:43.912 UTC [18229] LOG: background worker "logical replication launcher" (PID 18241) exited with exit code 1
2026-08-10 06:39:43.912 UTC [18233] LOG: shutting down
2026-08-10 06:39:44.060 UTC [18229] LOG: database system is shut down
2026-08-10T06:39:44Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50235 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:39:44 dist-test-slave-4tdj krb5kdc[19891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:39:44 dist-test-slave-4tdj krb5kdc[19891](info): set up 2 sockets
Aug 10 06:39:44 dist-test-slave-4tdj krb5kdc[19891](info): commencing operation
krb5kdc: starting...
W20260810 06:39:45.389472 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.006s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:39:45 dist-test-slave-4tdj krb5kdc[19891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:39:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:39:45Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.59.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.59.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.59.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/rangeradmin_127.31.59.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.59.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/rangeradmin_127.31.59.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.59.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.59.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.59.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/rangerlookup_127.31.59.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.59.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/rangerlookup_127.31.59.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.59.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.59.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.59.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/HTTP_127.31.59.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.59.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/HTTP_127.31.59.212@KRBTEST.COM.keytab.
I20260810 06:39:45.710665 31983 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskq68xXZ/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/postgres -l logfile start
2026-08-10 06:39:47.959 UTC [19944] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 06:39:47.960 UTC [19944] LOG: listening on IPv4 address "127.31.59.212", port 51463
2026-08-10 06:39:47.963 UTC [19944] LOG: listening on Unix socket "/tmp/.s.PGSQL.51463"
2026-08-10 06:39:47.984 UTC [19948] LOG: database system was shut down at 2026-08-10 06:39:46 UTC
2026-08-10 06:39:47.987 UTC [19944] LOG: database system is ready to accept connections
I20260810 06:39:49.073185 31983 mini_postgres.cc:96] Postgres bound to 51463
2026-08-10 06:39:49.093 UTC [19958] FATAL: database "slave" does not exist
127.31.59.212:51463 - accepting connections
I20260810 06:39:49.093997 31983 mini_ranger.cc:161] Starting Ranger...
I20260810 06:39:49.120144 31983 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 06:39:49.528342 31983 mini_ranger.cc:87] Created ranger Postgres database
I20260810 06:39:49.528466 31983 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-admin
2026-08-10 06:39:49,814 [I] DB FLAVOR :POSTGRES
2026-08-10 06:39:49,815 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:39:49,815 [I] Checking connection..
2026-08-10 06:39:49,815 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 06:39:50,072 [I] Checking connection passed.
2026-08-10 06:39:50,073 [I] --------- Verifying version history table ---------
2026-08-10 06:39:50,073 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:39:50,315 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:39:50,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:39:50,573 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:39:50,573 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 06:39:50,573 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 06:39:50.827 UTC [20054] WARNING: there is no transaction in progress
2026-08-10 06:39:50,833 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 06:39:50,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:39:51,090 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 06:39:51,091 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:39:51,091 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 06:39:51,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 06:39:51,577 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-4tdj.c.gcp-upstream.internal','N') ;"
2026-08-10 06:39:51,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:39:52,069 [I] Table x_portal_user does not exist in database ranger
2026-08-10 06:39:52,070 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:39:52,330 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 06:39:52,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:39:52,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:39:52,828 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 06:39:52,828 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 06:39:53.737 UTC [20229] WARNING: there is no transaction in progress
2026-08-10 06:39:53.752 UTC [20229] WARNING: there is no transaction in progress
2026-08-10 06:39:53.767 UTC [20229] WARNING: there is no transaction in progress
2026-08-10 06:39:53.781 UTC [20229] WARNING: there is no transaction in progress
2026-08-10 06:39:53.796 UTC [20229] WARNING: there is no transaction in progress
2026-08-10 06:39:53.813 UTC [20229] WARNING: there is no transaction in progress
2026-08-10 06:39:53.864 UTC [20229] WARNING: there is no transaction in progress
2026-08-10 06:39:53.870 UTC [20229] WARNING: there is no transaction in progress
2026-08-10 06:39:53.883 UTC [20229] WARNING: there is no transaction in progress
2026-08-10 06:39:53.892 UTC [20229] WARNING: there is no transaction in progress
2026-08-10 06:39:53.902 UTC [20229] WARNING: there is no transaction in progress
2026-08-10 06:39:53.915 UTC [20229] WARNING: there is no transaction in progress
2026-08-10 06:39:53.924 UTC [20229] WARNING: there is no transaction in progress
2026-08-10 06:39:53.929 UTC [20229] WARNING: there is no transaction in progress
2026-08-10 06:39:53.935 UTC [20229] WARNING: there is no transaction in progress
2026-08-10 06:39:53.941 UTC [20229] WARNING: there is no transaction in progress
2026-08-10 06:39:54,491 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 06:39:54,491 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 06:39:54,720 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 06:39:54,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:39:54,962 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 06:39:54,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:39:55,201 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 06:39:55,202 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:39:55,455 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:39:55,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-4tdj.c.gcp-upstream.internal';"
2026-08-10 06:39:55,950 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 06:39:55,951 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 06:39:56,202 [I] DB_PATCHES have already been applied
I20260810 06:39:56.208376 31983 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-admin:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskq68xXZ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 06:39:56.208483 31983 mini_ranger.cc:193] Using host: 127.31.59.212
I20260810 06:39:56.226130 31983 mini_ranger.cc:227] Ranger admin URL: http://127.31.59.212:41099
Aug 10, 2026 6:39:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:39:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:39:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 6:39:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 6:39:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 6:39:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 6:39:57 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 06:39:57.513775 31983 mini_ranger.cc:160] Time spent starting Ranger: real 8.420s user 0.007s sys 0.039s
Aug 10, 2026 6:40:17 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 6:40:17 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 6:40:17 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 6:40:17 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 6:40:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 6:40:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 6:40:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 6:40:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 6:40:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 6:40:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 6:40:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 6:40:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 6:40:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 6:40:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 6:40:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 6:40:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 6:40:17 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 6:40:18 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 06:40:18.535537 31983 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.59.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.59.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.59.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/rangerkms_127.31.59.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.59.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/rangerkms_127.31.59.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.59.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.59.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.59.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/HTTP_127.31.59.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.59.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/HTTP_127.31.59.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 06:40:18 dist-test-slave-4tdj krb5kdc[19891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344018, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 06:40:18.705679 31983 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 06:40:18.728457 31983 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 06:40:19.437052 31983 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 06:40:19.437168 31983 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-kms
I20260810 06:40:19.437213 31983 mini_ranger_kms.cc:227] Using postgres at 127.31.59.212:51463
2026-08-10 06:40:19,790 [I] DB FLAVOR :POSTGRES
2026-08-10 06:40:19,790 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:40:19,790 [I] Checking connection
2026-08-10 06:40:19,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 06:40:20,044 [I] connection success
2026-08-10 06:40:20,044 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 06:40:20,044 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 06:40:20,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 06:40:20,271 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 06:40:20,271 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:40:20,271 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 06:40:20,271 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.59.212:51463/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 06:40:20,548 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 06:40:20.978230 31983 mini_ranger_kms.cc:310] Created kms service
I20260810 06:40:21.141551 31983 mini_ranger_kms.cc:326] Created kudu user
I20260810 06:40:21.182116 31983 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 06:40:21 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344018, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:40:21.548563 31983 mini_ranger_kms.cc:384] Added ranger policy
I20260810 06:40:21.548889 31983 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-kms:/tmp/dist-test-taskq68xXZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskq68xXZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 06:40:21.548951 31983 mini_ranger_kms.cc:240] Using host: 127.31.59.212
I20260810 06:40:21.569458 31983 mini_ranger_kms.cc:276] Ranger KMS PID: 20519
I20260810 06:40:21.569651 31983 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.59.212:35477
Aug 10, 2026 6:40:22 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:40:22 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:40:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 6:40:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 6:40:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 6:40:22 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 06:40:23 dist-test-slave-4tdj krb5kdc[19891](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786344023, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.59.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:24.183079 31983 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.477s user 0.068s sys 0.314s
I20260810 06:40:24.183328 31983 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 06:40:24.183480 31983 mini_ranger_kms.cc:401] 127.31.59.212:35477/kms/v1/keys
Aug 10 06:40:24 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344023, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.59.212@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
Aug 10 06:40:24 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344023, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.59.212@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
Aug 10 06:40:26 dist-test-slave-4tdj krb5kdc[19891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:40:26.952189 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40907
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:40907
--ranger_config_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.212:35477/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:27.295284 20599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:27.295570 20599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:27.295711 20599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:27.302838 20599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:27.302955 20599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:27.303035 20599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:27.303108 20599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:27.303174 20599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:27.313328 20599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.59.212:35477/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:40907
--ranger_config_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:27.314692 20599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:27.316056 20599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:27.327618 20609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:27.327913 20606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:27.328004 20607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:27.328578 20599 server_base.cc:1034] running on GCE node
I20260810 06:40:27.329308 20599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:27.330674 20599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:27.331879 20599 hybrid_clock.cc:648] HybridClock initialized: now 1786344027331833 us; error 39 us; skew 500 ppm
I20260810 06:40:27.332204 20599 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:27 dist-test-slave-4tdj krb5kdc[19891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344027, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:27.341271 20599 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:40:27.342788 20599 webserver.cc:460] Webserver started at http://127.31.59.254:33121/ using document root <none> and password file <none>
I20260810 06:40:27.343415 20599 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:27.343538 20599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:27.343842 20599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:40:27 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344027, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:40:27.400522 20599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "4c8a90d6914048b9b95f308993c1b450"
format_stamp: "Formatted at 2026-08-10 06:40:27 on dist-test-slave-4tdj"
server_key: "2195bcb16c0ad63a079d62453b2f6592"
server_key_iv: "077186c27accbc68c898e5271dd58734"
server_key_version: "kuduclusterkey@0"
I20260810 06:40:27.401746 20599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "4c8a90d6914048b9b95f308993c1b450"
format_stamp: "Formatted at 2026-08-10 06:40:27 on dist-test-slave-4tdj"
server_key: "2195bcb16c0ad63a079d62453b2f6592"
server_key_iv: "077186c27accbc68c898e5271dd58734"
server_key_version: "kuduclusterkey@0"
I20260810 06:40:27.409127 20599 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260810 06:40:27.428663 20617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:27.430126 20599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:40:27.430285 20599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "4c8a90d6914048b9b95f308993c1b450"
format_stamp: "Formatted at 2026-08-10 06:40:27 on dist-test-slave-4tdj"
server_key: "2195bcb16c0ad63a079d62453b2f6592"
server_key_iv: "077186c27accbc68c898e5271dd58734"
server_key_version: "kuduclusterkey@0"
I20260810 06:40:27.430564 20599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:27.456918 20599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:27.460865 20599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:27.461143 20599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:27.478809 20599 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:40907
I20260810 06:40:27.478853 20700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:40907 every 8 connection(s)
I20260810 06:40:27.480158 20599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:40:27.483855 20702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:27.490633 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 20599
I20260810 06:40:27.490855 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:40:27.491695 31983 external_mini_cluster.cc:1428] Setting key 0bbf969b4620fc102db7486f11054fb8
I20260810 06:40:27.494844 20702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450: Bootstrap starting.
I20260810 06:40:27.498229 20702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:27.499361 20702 log.cc:824] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:27.502331 20702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450: No bootstrap required, opened a new log
Aug 10 06:40:27 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:40:27.510986 20702 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8a90d6914048b9b95f308993c1b450" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40907 } }
I20260810 06:40:27.511404 20702 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:27.511546 20702 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c8a90d6914048b9b95f308993c1b450, State: Initialized, Role: FOLLOWER
I20260810 06:40:27.512212 20702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8a90d6914048b9b95f308993c1b450" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40907 } }
I20260810 06:40:27.512483 20702 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:40:27.512633 20702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:40:27.512806 20702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:27.514367 20706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:27.493708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46914 (local address 127.31.59.254:40907)
0810 06:40:27.494502 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:27.494513 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:27.494555 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:40:27.496260 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:27.496282 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:27.496645 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:27.496949 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:27.498590 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:27.500012 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:27.501700 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:27.502380 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:27.502550 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:27.506202 (+ 3652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:27.506213 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:27.506235 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:27.506284 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:27.510042 (+ 3758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:27.510705 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:27.510714 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:27.510721 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:27.510899 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:27.511220 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:27.511225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:27.511228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:27.511971 (+ 743us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:27.512245 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:27.512635 (+ 390us) server_negotiation.cc:299] Negotiation successful
0810 06:40:27.513039 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":349,"threads_started":1}
I20260810 06:40:27.515554 20702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8a90d6914048b9b95f308993c1b450" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40907 } }
I20260810 06:40:27.516027 20702 leader_election.cc:302] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c8a90d6914048b9b95f308993c1b450; no voters:
I20260810 06:40:27.516664 20702 leader_election.cc:288] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:40:27.516775 20708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:40:27.517501 20708 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [term 1 LEADER]: Becoming Leader. State: Replica: 4c8a90d6914048b9b95f308993c1b450, State: Running, Role: LEADER
I20260810 06:40:27.518074 20708 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8a90d6914048b9b95f308993c1b450" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40907 } }
I20260810 06:40:27.518968 20702 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:27.520910 20710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c8a90d6914048b9b95f308993c1b450. Latest consensus state: current_term: 1 leader_uuid: "4c8a90d6914048b9b95f308993c1b450" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8a90d6914048b9b95f308993c1b450" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40907 } } }
I20260810 06:40:27.521322 20710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:27.521734 20709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c8a90d6914048b9b95f308993c1b450" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8a90d6914048b9b95f308993c1b450" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40907 } } }
I20260810 06:40:27.522023 20709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:27.531266 20719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:40:27.535877 20719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:40:27.540869 20702 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 06:40:27.552217 20719 catalog_manager.cc:1347] Generated new cluster ID: ac5e03500d1047bb9345f29fd3645bd3
I20260810 06:40:27.552426 20719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:40:27.571282 20719 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:40:27.573288 20719 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:40:27.590284 20719 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450: Generated new TSK 0
I20260810 06:40:27.591481 20719 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
I20260810 06:40:28.333891 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.212:35477/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40907
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:40:28 dist-test-slave-4tdj krb5kdc[19891](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786344028, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 06:40:28.715128 20771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:28.715405 20771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:28.715579 20771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:28.723235 20771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:28.723358 20771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:28.723546 20771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:40:28.733790 20771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.59.212:35477/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40907
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:28.735329 20771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:28.736716 20771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:28.753549 20790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:28.754480 20793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:28.759521 20795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:28.759773 20771 server_base.cc:1034] running on GCE node
I20260810 06:40:28.760521 20771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:28.761596 20771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:28.762826 20771 hybrid_clock.cc:648] HybridClock initialized: now 1786344028762773 us; error 50 us; skew 500 ppm
I20260810 06:40:28.763155 20771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:28 dist-test-slave-4tdj krb5kdc[19891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344028, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:28.766815 20771 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:40:28.768982 20771 webserver.cc:460] Webserver started at http://127.31.59.193:35483/ using document root <none> and password file <none>
I20260810 06:40:28.769685 20771 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:28.769804 20771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:28.770174 20771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:40:28 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344028, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:40:28.792142 20771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "0944caa46362487fb58b91ec2aa2d8f9"
format_stamp: "Formatted at 2026-08-10 06:40:28 on dist-test-slave-4tdj"
server_key: "280c859e0b3c2dc343ffe41a47d97f06"
server_key_iv: "be35ea03cc2a719cc36de28133dad40f"
server_key_version: "kuduclusterkey@0"
I20260810 06:40:28.793969 20771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "0944caa46362487fb58b91ec2aa2d8f9"
format_stamp: "Formatted at 2026-08-10 06:40:28 on dist-test-slave-4tdj"
server_key: "280c859e0b3c2dc343ffe41a47d97f06"
server_key_iv: "be35ea03cc2a719cc36de28133dad40f"
server_key_version: "kuduclusterkey@0"
I20260810 06:40:28.801520 20771 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.006s
I20260810 06:40:28.848594 20818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:28.853734 20771 fs_manager.cc:725] Time spent opening block manager: real 0.025s user 0.014s sys 0.006s
I20260810 06:40:28.854055 20771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "0944caa46362487fb58b91ec2aa2d8f9"
format_stamp: "Formatted at 2026-08-10 06:40:28 on dist-test-slave-4tdj"
server_key: "280c859e0b3c2dc343ffe41a47d97f06"
server_key_iv: "be35ea03cc2a719cc36de28133dad40f"
server_key_version: "kuduclusterkey@0"
I20260810 06:40:28.854465 20771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:28.896795 20771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:28.902161 20771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:28.902638 20771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:28.904775 20771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:40:28.907162 20771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:40:28.907316 20771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260810 06:40:28.907531 20771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:40:28.907675 20771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:40:29 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344028, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:40:29.273322 20987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:34459 every 8 connection(s)
I20260810 06:40:29.273701 20771 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:34459
I20260810 06:40:29.275699 20771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:40:29.281081 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 20771
I20260810 06:40:29.281296 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:40:29.282150 31983 external_mini_cluster.cc:1428] Setting key 0226afb4211607e969d5ce306df3552c
Aug 10 06:40:29 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344028, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:40:29.322635 20994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:29.284896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:38443 (local address 127.31.59.254:40907)
0810 06:40:29.303734 (+ 18838us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:29.303740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:29.303758 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:40:29.303815 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:29.303820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:29.303904 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:29.304013 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:29.306139 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:29.307246 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:29.309411 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:29.309965 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:29.310051 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:29.315139 (+ 5088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:29.315147 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:29.315151 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:29.315193 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:29.318787 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:29.319751 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:29.319757 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:29.319761 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:29.319924 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:29.320503 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:29.320508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:29.320511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:29.320857 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:29.320968 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:29.321963 (+ 995us) server_negotiation.cc:299] Negotiation successful
0810 06:40:29.322202 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18720,"thread_start_us":18620,"threads_started":1}
WARNING: no policy specified for kudu/127.31.59.194@KRBTEST.COM; defaulting to no policy
I20260810 06:40:29.325172 20993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:29.284303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40907 (local address 127.31.59.193:38443)
0810 06:40:29.294975 (+ 10672us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:29.295041 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:29.297355 (+ 2314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:29.304351 (+ 6996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:29.304365 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:29.305026 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:29.305926 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:29.305955 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:29.307434 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:29.307439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:29.309250 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:29.309265 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:29.310913 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:29.310924 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:29.311225 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:29.312010 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:29.312041 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:29.314846 (+ 2805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:29.318970 (+ 4124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:29.318979 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:29.318997 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:29.319608 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:29.320091 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:29.320096 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:29.320101 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:29.320376 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:29.321097 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:29.321104 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:29.321529 (+ 425us) client_negotiation.cc:769] Sending connection context
0810 06:40:29.323128 (+ 1599us) client_negotiation.cc:240] Negotiation successful
0810 06:40:29.323703 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10335,"thread_start_us":10082,"threads_started":1}
I20260810 06:40:29.327296 20988 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40907
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.194@KRBTEST.COM" created.
I20260810 06:40:29.327643 20988 heartbeater.cc:464] Registering TS with master...
I20260810 06:40:29.328509 20988 heartbeater.cc:511] Master 127.31.59.254:40907 requested a full tablet report, sending...
I20260810 06:40:29.333549 20642 ts_manager.cc:194] Registered new tserver with Master: 0944caa46362487fb58b91ec2aa2d8f9 (127.31.59.193:34459)
I20260810 06:40:29.335670 20642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:38443
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
I20260810 06:40:29.371398 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.212:35477/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40907
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:40:29 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344028, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
W20260810 06:40:29.837471 20998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:29.837765 20998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:29.837906 20998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:29.845055 20998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:29.845175 20998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:29.845336 20998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:40:29.855050 20998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.59.212:35477/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40907
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:29.856295 20998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:29.857590 20998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:29.871485 21016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:29.871313 21017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:29.871794 21021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:29.872452 20998 server_base.cc:1034] running on GCE node
I20260810 06:40:29.873044 20998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:29.873967 20998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:29.875178 20998 hybrid_clock.cc:648] HybridClock initialized: now 1786344029875126 us; error 48 us; skew 500 ppm
I20260810 06:40:29.875474 20998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:29 dist-test-slave-4tdj krb5kdc[19891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344029, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:29.878602 20998 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:40:29.880147 20998 webserver.cc:460] Webserver started at http://127.31.59.194:42945/ using document root <none> and password file <none>
I20260810 06:40:29.880797 20998 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:29.880919 20998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:29.881244 20998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:40:29 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344029, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:40:29.898133 20998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "da5a85c35f0c4c54949b96adb1ecc4c3"
format_stamp: "Formatted at 2026-08-10 06:40:29 on dist-test-slave-4tdj"
server_key: "0412af803d10651f663cc256b933684e"
server_key_iv: "b241a894a994c207c7b12c1053615875"
server_key_version: "kuduclusterkey@0"
I20260810 06:40:29.899025 20998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "da5a85c35f0c4c54949b96adb1ecc4c3"
format_stamp: "Formatted at 2026-08-10 06:40:29 on dist-test-slave-4tdj"
server_key: "0412af803d10651f663cc256b933684e"
server_key_iv: "b241a894a994c207c7b12c1053615875"
server_key_version: "kuduclusterkey@0"
I20260810 06:40:29.904601 20998 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 06:40:29.922603 21031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:29.924119 20998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 06:40:29.924326 20998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "da5a85c35f0c4c54949b96adb1ecc4c3"
format_stamp: "Formatted at 2026-08-10 06:40:29 on dist-test-slave-4tdj"
server_key: "0412af803d10651f663cc256b933684e"
server_key_iv: "b241a894a994c207c7b12c1053615875"
server_key_version: "kuduclusterkey@0"
I20260810 06:40:29.924562 20998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:29.953366 20998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:29.957720 20998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:29.958019 20998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:29.959122 20998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:40:29.960742 20998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:40:29.960821 20998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:29.960939 20998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:40:29.960994 20998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:30.009207 20998 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:35525
I20260810 06:40:30.009212 21210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:35525 every 8 connection(s)
I20260810 06:40:30.010633 20998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:40:30.016783 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 20998
I20260810 06:40:30.016974 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:40:30.017557 31983 external_mini_cluster.cc:1428] Setting key 2e3885aa173a4f354c16e87c93194264
Aug 10 06:40:30 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344029, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:40:30.058928 21214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:30.013724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:40775 (local address 127.31.59.254:40907)
0810 06:40:30.032693 (+ 18969us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:30.032699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:30.036665 (+ 3966us) server_negotiation.cc:407] Connection header received
0810 06:40:30.040055 (+ 3390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:30.040062 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:30.040171 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:30.040295 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:30.042376 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:30.043592 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:30.045880 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:30.046541 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:30.046665 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:30.051475 (+ 4810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:30.051483 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:30.051488 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:30.051534 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:30.055106 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:30.056095 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:30.056100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:30.056104 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:30.056261 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:30.056788 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:30.056793 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:30.056797 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:30.057163 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:30.057282 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:30.058266 (+ 984us) server_negotiation.cc:299] Negotiation successful
0810 06:40:30.058527 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18823,"thread_start_us":18969,"threads_started":1}
I20260810 06:40:30.061599 21215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:30.014536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40907 (local address 127.31.59.194:40775)
0810 06:40:30.036432 (+ 21896us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:30.036500 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:30.039690 (+ 3190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:30.040599 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:30.040611 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:30.041214 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:30.042146 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:30.042175 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:30.043782 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:30.043787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:30.045719 (+ 1932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:30.045734 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:30.046986 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:30.046994 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:30.047298 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:30.048123 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:30.048154 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:30.051171 (+ 3017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:30.055289 (+ 4118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:30.055298 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:30.055317 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:30.055930 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:30.056394 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:30.056399 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:30.056402 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:30.056639 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:30.057391 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:30.057400 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:30.057817 (+ 417us) client_negotiation.cc:769] Sending connection context
0810 06:40:30.059415 (+ 1598us) client_negotiation.cc:240] Negotiation successful
0810 06:40:30.059975 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21561,"thread_start_us":21305,"threads_started":1}
I20260810 06:40:30.063573 21211 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40907
I20260810 06:40:30.063915 21211 heartbeater.cc:464] Registering TS with master...
I20260810 06:40:30.064793 21211 heartbeater.cc:511] Master 127.31.59.254:40907 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.59.195@KRBTEST.COM; defaulting to no policy
I20260810 06:40:30.067306 20642 ts_manager.cc:194] Registered new tserver with Master: da5a85c35f0c4c54949b96adb1ecc4c3 (127.31.59.194:35525)
I20260810 06:40:30.068465 20642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:40775
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
I20260810 06:40:30.105422 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.212:35477/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40907
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:40:30.339501 20988 heartbeater.cc:503] Master 127.31.59.254:40907 was elected leader, sending a full tablet report...
W20260810 06:40:30.482185 21221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:30.482544 21221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:30.482692 21221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:30.492369 21221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:30.492472 21221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:30.492599 21221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:40:30.502071 21221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.59.212:35477/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40907
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:30.503264 21221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:30.504508 21221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:30.517302 21230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:30.517262 21233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:30.517349 21231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:30.518064 21221 server_base.cc:1034] running on GCE node
I20260810 06:40:30.518666 21221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:30.519594 21221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:30.520787 21221 hybrid_clock.cc:648] HybridClock initialized: now 1786344030520745 us; error 32 us; skew 500 ppm
I20260810 06:40:30.521075 21221 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:30 dist-test-slave-4tdj krb5kdc[19891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344030, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:30.524034 21221 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:40:30.525380 21221 webserver.cc:460] Webserver started at http://127.31.59.195:38903/ using document root <none> and password file <none>
I20260810 06:40:30.526002 21221 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:30.526118 21221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:30.526453 21221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:40:30 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344030, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:40:30.541296 21221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "f710875ed20742ed83f2039fc4189030"
format_stamp: "Formatted at 2026-08-10 06:40:30 on dist-test-slave-4tdj"
server_key: "80de8bb840b17979c776505b33f20dda"
server_key_iv: "eadad6279053954b6967642ac2894ca1"
server_key_version: "kuduclusterkey@0"
I20260810 06:40:30.542150 21221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "f710875ed20742ed83f2039fc4189030"
format_stamp: "Formatted at 2026-08-10 06:40:30 on dist-test-slave-4tdj"
server_key: "80de8bb840b17979c776505b33f20dda"
server_key_iv: "eadad6279053954b6967642ac2894ca1"
server_key_version: "kuduclusterkey@0"
I20260810 06:40:30.547799 21221 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 06:40:30.566923 21245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:30.568516 21221 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.007s sys 0.001s
I20260810 06:40:30.568687 21221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "f710875ed20742ed83f2039fc4189030"
format_stamp: "Formatted at 2026-08-10 06:40:30 on dist-test-slave-4tdj"
server_key: "80de8bb840b17979c776505b33f20dda"
server_key_iv: "eadad6279053954b6967642ac2894ca1"
server_key_version: "kuduclusterkey@0"
I20260810 06:40:30.568908 21221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:30.584076 21221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:30.587460 21221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:30.587726 21221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:30.588606 21221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:40:30.589915 21221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:40:30.589994 21221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:30.590118 21221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:40:30.590176 21221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:30.620772 21221 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:43537
I20260810 06:40:30.620836 21421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:43537 every 8 connection(s)
I20260810 06:40:30.622088 21221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:40:30.627619 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 21221
I20260810 06:40:30.627774 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:40:30.628338 31983 external_mini_cluster.cc:1428] Setting key aaf4a1926a9b5353ed5c7a7119d827f0
Aug 10 06:40:30 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344030, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:40:30.641896 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:30.624526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:51989 (local address 127.31.59.254:40907)
0810 06:40:30.625207 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:30.625228 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:30.626270 (+ 1042us) server_negotiation.cc:407] Connection header received
0810 06:40:30.628109 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:30.628116 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:30.628212 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:30.628328 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:30.630050 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:30.630991 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:30.632822 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:30.633246 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:30.633335 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:30.636647 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:30.636651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:30.636655 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:30.636690 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:30.639370 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:30.639974 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:30.639977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:30.639979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:30.640094 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:30.640399 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:30.640402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:30.640404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:30.640702 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:30.640845 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:30.641415 (+ 570us) server_negotiation.cc:299] Negotiation successful
0810 06:40:30.641571 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":289,"threads_started":1}
I20260810 06:40:30.642845 21427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:30.625101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40907 (local address 127.31.59.195:51989)
0810 06:40:30.626111 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:30.626157 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:30.627853 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:30.628613 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:30.628622 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:30.629105 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:30.629886 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:30.629906 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:30.631136 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:30.631142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:30.632709 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:30.632721 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:30.633407 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:30.633411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:30.633654 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:30.634258 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:30.634284 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:30.636463 (+ 2179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:30.639467 (+ 3004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:30.639474 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:30.639485 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:30.639886 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:30.640167 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:30.640170 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:30.640172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:30.640328 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:30.640807 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:30.640813 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:30.641086 (+ 273us) client_negotiation.cc:769] Sending connection context
0810 06:40:30.641300 (+ 214us) client_negotiation.cc:240] Negotiation successful
0810 06:40:30.641662 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":416,"threads_started":1}
I20260810 06:40:30.644255 21422 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40907
I20260810 06:40:30.644541 21422 heartbeater.cc:464] Registering TS with master...
I20260810 06:40:30.645195 21422 heartbeater.cc:511] Master 127.31.59.254:40907 requested a full tablet report, sending...
I20260810 06:40:30.646605 20641 ts_manager.cc:194] Registered new tserver with Master: f710875ed20742ed83f2039fc4189030 (127.31.59.195:43537)
I20260810 06:40:30.647518 20641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:51989
I20260810 06:40:30.655766 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:40:30.657548 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 20771
Aug 10 06:40:30 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:40:30.675779 21429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:30.658394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59254 (local address 127.31.59.193:34459)
0810 06:40:30.659219 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:30.659226 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:30.659249 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:40:30.659309 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:30.659315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:30.659518 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:30.659642 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:30.661137 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:30.662549 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:30.664219 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:30.664856 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:30.664951 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:30.669433 (+ 4482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:30.669445 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:30.669458 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:30.669490 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:30.672885 (+ 3395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:30.673802 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:30.673808 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:30.673810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:30.673963 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:30.674416 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:30.674429 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:30.674431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:30.674740 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:30.675005 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:30.675247 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 06:40:30.675440 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":443,"threads_started":1}
I20260810 06:40:30.677009 20983 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:30.759786 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 20771
I20260810 06:40:30.781721 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 20998
Aug 10 06:40:30 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.194@KRBTEST.COM
I20260810 06:40:30.797530 21431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:30.782838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56230 (local address 127.31.59.194:35525)
0810 06:40:30.783401 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:30.783408 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:30.783433 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:40:30.783494 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:30.783500 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:30.783729 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:30.783868 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:30.785079 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:30.786084 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:30.787327 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:30.787882 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:30.787977 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:30.792251 (+ 4274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:30.792262 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:30.792276 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:30.792317 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:30.795083 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:30.795740 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:30.795747 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:30.795749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:30.795883 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:30.796212 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:30.796215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:30.796217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:30.796576 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:30.796805 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:30.797050 (+ 245us) server_negotiation.cc:299] Negotiation successful
0810 06:40:30.797251 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":274,"threads_started":1}
I20260810 06:40:30.798813 21203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:30.879730 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 20998
I20260810 06:40:30.907629 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 21221
Aug 10 06:40:30 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.195@KRBTEST.COM
I20260810 06:40:30.924127 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:30.908584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60990 (local address 127.31.59.195:43537)
0810 06:40:30.909209 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:30.909216 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:30.909246 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:40:30.909303 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:30.909310 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:30.909532 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:30.909676 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:30.911006 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:30.912007 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:30.913546 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:30.914080 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:30.914194 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:30.918466 (+ 4272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:30.918477 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:30.918491 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:30.918538 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:30.921438 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:30.922079 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:30.922086 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:30.922088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:30.922218 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:30.922588 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:30.922592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:30.922594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:30.922945 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:30.923252 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:30.923542 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 06:40:30.923792 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":311,"threads_started":1}
I20260810 06:40:30.925437 21415 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:31.008319 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 21221
I20260810 06:40:31.029254 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 20599
I20260810 06:40:31.030527 20690 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:31.135708 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 20599
I20260810 06:40:31.165021 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40907
--webserver_interface=127.31.59.254
--webserver_port=33121
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:40907
--ranger_config_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.212:35477/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.59.212:35477/kms with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:31.484313 21437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:31.484601 21437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:31.484742 21437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:31.491652 21437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:31.491758 21437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:31.491838 21437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:31.491915 21437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:31.491981 21437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:31.501605 21437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.59.212:35477/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:40907
--ranger_config_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=33121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:31.502856 21437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:31.504130 21437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:31.515147 21447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:31.515326 21450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:31.515559 21453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:31.516279 21437 server_base.cc:1034] running on GCE node
I20260810 06:40:31.517055 21437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:31.518491 21437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:31.519744 21437 hybrid_clock.cc:648] HybridClock initialized: now 1786344031519685 us; error 43 us; skew 500 ppm
I20260810 06:40:31.520092 21437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:31 dist-test-slave-4tdj krb5kdc[19891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344031, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:31.523147 21437 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:40:31.524739 21437 webserver.cc:460] Webserver started at http://127.31.59.254:33121/ using document root <none> and password file <none>
I20260810 06:40:31.525437 21437 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:31.525585 21437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:40:31 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344031, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:40:31.605546 21437 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260810 06:40:31.608659 21464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:31.610018 21437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:40:31.610179 21437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "4c8a90d6914048b9b95f308993c1b450"
format_stamp: "Formatted at 2026-08-10 06:40:27 on dist-test-slave-4tdj"
server_key: "2195bcb16c0ad63a079d62453b2f6592"
server_key_iv: "077186c27accbc68c898e5271dd58734"
server_key_version: "kuduclusterkey@0"
I20260810 06:40:31.611133 21437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:31.628594 21437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:31.632313 21437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:31.632580 21437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:31.649613 21437 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:40907
I20260810 06:40:31.649618 21549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:40907 every 8 connection(s)
I20260810 06:40:31.651087 21437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:40:31.657872 21550 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:40:31.658448 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 21437
I20260810 06:40:31.659559 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.212:35477/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:34459
--local_ip_for_outbound_sockets=127.31.59.193
--tserver_master_addrs=127.31.59.254:40907
--webserver_port=35483
--webserver_interface=127.31.59.193
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.59.212:35477/kms with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:40:31.662472 21550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450: Bootstrap starting.
I20260810 06:40:31.682412 21550 log.cc:824] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:31.690994 21550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:40:31.691484 21550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450: Bootstrap complete.
I20260810 06:40:31.699951 21550 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8a90d6914048b9b95f308993c1b450" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40907 } }
I20260810 06:40:31.700768 21550 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c8a90d6914048b9b95f308993c1b450, State: Initialized, Role: FOLLOWER
I20260810 06:40:31.701345 21550 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8a90d6914048b9b95f308993c1b450" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40907 } }
I20260810 06:40:31.701539 21550 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:40:31.701645 21550 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:40:31.701804 21550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:40:31.705483 21550 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8a90d6914048b9b95f308993c1b450" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40907 } }
I20260810 06:40:31.705924 21550 leader_election.cc:302] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c8a90d6914048b9b95f308993c1b450; no voters:
I20260810 06:40:31.706524 21550 leader_election.cc:288] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:40:31.706660 21554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:40:31.707707 21554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [term 2 LEADER]: Becoming Leader. State: Replica: 4c8a90d6914048b9b95f308993c1b450, State: Running, Role: LEADER
I20260810 06:40:31.708235 21554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8a90d6914048b9b95f308993c1b450" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40907 } }
I20260810 06:40:31.708583 21550 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:31.711427 21555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4c8a90d6914048b9b95f308993c1b450" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8a90d6914048b9b95f308993c1b450" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40907 } } }
I20260810 06:40:31.711782 21555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:31.713977 21556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c8a90d6914048b9b95f308993c1b450. Latest consensus state: current_term: 2 leader_uuid: "4c8a90d6914048b9b95f308993c1b450" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8a90d6914048b9b95f308993c1b450" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40907 } } }
I20260810 06:40:31.714263 21556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450 [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:31.715662 21563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:40:31.721022 21563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:40:31.722368 21563 catalog_manager.cc:1259] Loaded cluster ID: ac5e03500d1047bb9345f29fd3645bd3
I20260810 06:40:31.722458 21563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:40:31.726414 21563 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:40:31.727777 21563 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450: Loaded TSK: 0
I20260810 06:40:31.729271 21563 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:40:31.736032 21550 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260810 06:40:32.124506 21552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:32.125023 21552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:32.125242 21552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:32.136660 21552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:32.136793 21552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:32.136970 21552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:40:32.152215 21552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.59.212:35477/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:34459
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=35483
--tserver_master_addrs=127.31.59.254:40907
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:32.154048 21552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:32.155807 21552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:32.182775 21606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:32.190554 21552 server_base.cc:1034] running on GCE node
W20260810 06:40:32.191675 21610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:32.191808 21605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:32.192725 21552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:32.194315 21552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:32.195600 21552 hybrid_clock.cc:648] HybridClock initialized: now 1786344032195517 us; error 53 us; skew 500 ppm
I20260810 06:40:32.196015 21552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:32 dist-test-slave-4tdj krb5kdc[19891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344032, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:32.202407 21552 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:40:32.221272 21552 webserver.cc:460] Webserver started at http://127.31.59.193:35483/ using document root <none> and password file <none>
I20260810 06:40:32.222429 21552 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:32.222630 21552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:40:32 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344032, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:40:32.261189 21552 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 06:40:32.265079 21619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:32.267379 21552 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 06:40:32.267573 21552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "0944caa46362487fb58b91ec2aa2d8f9"
format_stamp: "Formatted at 2026-08-10 06:40:28 on dist-test-slave-4tdj"
server_key: "280c859e0b3c2dc343ffe41a47d97f06"
server_key_iv: "be35ea03cc2a719cc36de28133dad40f"
server_key_version: "kuduclusterkey@0"
I20260810 06:40:32.268998 21552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:32.292146 21552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:32.297215 21552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:32.297523 21552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:32.298750 21552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:40:32.300462 21552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:40:32.300590 21552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:32.300699 21552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:40:32.300770 21552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:32.362962 21552 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:34459
I20260810 06:40:32.363018 21794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:34459 every 8 connection(s)
I20260810 06:40:32.364400 21552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:40:32.368472 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 21552
I20260810 06:40:32.369388 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.212:35477/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:35525
--local_ip_for_outbound_sockets=127.31.59.194
--tserver_master_addrs=127.31.59.254:40907
--webserver_port=42945
--webserver_interface=127.31.59.194
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.59.212:35477/kms with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:40:32 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344032, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:40:32.401609 21801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:32.369205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:52697 (local address 127.31.59.254:40907)
0810 06:40:32.374624 (+ 5419us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:32.374644 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:32.374697 (+ 53us) server_negotiation.cc:407] Connection header received
0810 06:40:32.375119 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:32.375137 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:32.375645 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:32.376047 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:32.379375 (+ 3328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:32.380859 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:32.383056 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:32.383996 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:32.384158 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:32.392325 (+ 8167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:32.392337 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:32.392357 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:32.392414 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:32.396231 (+ 3817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:32.397133 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:32.397143 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:32.397152 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:32.397363 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:32.397884 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:32.397890 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:32.397893 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:32.398627 (+ 734us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:32.398829 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:32.399939 (+ 1110us) server_negotiation.cc:299] Negotiation successful
0810 06:40:32.400358 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5126,"thread_start_us":4969,"threads_started":1}
I20260810 06:40:32.401629 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:32.368298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40907 (local address 127.31.59.193:52697)
0810 06:40:32.370710 (+ 2412us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:32.370770 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:32.372930 (+ 2160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:32.377767 (+ 4837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:32.377779 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:32.378385 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:32.379132 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:32.379157 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:32.381026 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:32.381032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:32.382896 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:32.382910 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:32.384555 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:32.384561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:32.384843 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:32.388671 (+ 3828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:32.388701 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:32.392003 (+ 3302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:32.396398 (+ 4395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:32.396406 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:32.396421 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:32.396940 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:32.397496 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:32.397501 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:32.397504 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:32.397748 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:32.398900 (+ 1152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:32.398908 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:32.399340 (+ 432us) client_negotiation.cc:769] Sending connection context
0810 06:40:32.399660 (+ 320us) client_negotiation.cc:240] Negotiation successful
0810 06:40:32.400254 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2085,"thread_start_us":1872,"threads_started":1}
I20260810 06:40:32.405464 21795 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40907
I20260810 06:40:32.405815 21795 heartbeater.cc:464] Registering TS with master...
I20260810 06:40:32.406780 21795 heartbeater.cc:511] Master 127.31.59.254:40907 requested a full tablet report, sending...
W20260810 06:40:32.410267 21795 heartbeater.cc:644] Failed to heartbeat to 127.31.59.254:40907 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 06:40:32.410533 21795 heartbeater.cc:464] Registering TS with master...
I20260810 06:40:32.410822 21795 heartbeater.cc:511] Master 127.31.59.254:40907 requested a full tablet report, sending...
I20260810 06:40:32.412156 21795 heartbeater.cc:464] Registering TS with master...
I20260810 06:40:32.412437 21795 heartbeater.cc:511] Master 127.31.59.254:40907 requested a full tablet report, sending...
W20260810 06:40:32.413532 21795 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:40:32.744794 21802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:32.745069 21802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:32.745205 21802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:32.752800 21802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:32.752992 21802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:32.753203 21802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:40:32.765031 21802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.59.212:35477/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:35525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=42945
--tserver_master_addrs=127.31.59.254:40907
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:32.766455 21802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:32.767701 21802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:32.783289 21826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:32.784950 21828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:32.785521 21824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:32.786078 21802 server_base.cc:1034] running on GCE node
I20260810 06:40:32.786892 21802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:32.788131 21802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:32.789422 21802 hybrid_clock.cc:648] HybridClock initialized: now 1786344032789311 us; error 91 us; skew 500 ppm
I20260810 06:40:32.789801 21802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:32 dist-test-slave-4tdj krb5kdc[19891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344032, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:32.794080 21802 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:40:32.797363 21802 webserver.cc:460] Webserver started at http://127.31.59.194:42945/ using document root <none> and password file <none>
I20260810 06:40:32.798257 21802 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:32.798453 21802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:40:32 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344032, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:40:32.824882 21802 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.002s sys 0.002s
I20260810 06:40:32.838939 21838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:32.841006 21802 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.000s sys 0.012s
I20260810 06:40:32.841305 21802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "da5a85c35f0c4c54949b96adb1ecc4c3"
format_stamp: "Formatted at 2026-08-10 06:40:29 on dist-test-slave-4tdj"
server_key: "0412af803d10651f663cc256b933684e"
server_key_iv: "b241a894a994c207c7b12c1053615875"
server_key_version: "kuduclusterkey@0"
I20260810 06:40:32.842896 21802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:32.931788 21802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:32.937098 21802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:32.937490 21802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:32.938885 21802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
Aug 10 06:40:32 dist-test-slave-4tdj krb5kdc[19891](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786344032, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:32.940838 21802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:40:32.940954 21802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:32.941079 21802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:40:32.941164 21802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:33.033885 21802 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:35525
I20260810 06:40:33.033950 21998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:35525 every 8 connection(s)
I20260810 06:40:33.035677 21802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:40:33.045910 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 21802
I20260810 06:40:33.046751 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.59.212:35477/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:43537
--local_ip_for_outbound_sockets=127.31.59.195
--tserver_master_addrs=127.31.59.254:40907
--webserver_port=38903
--webserver_interface=127.31.59.195
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.59.212:35477/kms with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:40:33 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344032, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:40:33.063048 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:33.038718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:42331 (local address 127.31.59.254:40907)
0810 06:40:33.039802 (+ 1084us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:33.039809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:33.042574 (+ 2765us) server_negotiation.cc:407] Connection header received
0810 06:40:33.043955 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:33.043961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:33.044049 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:33.044146 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:33.046143 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.047580 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.049718 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.050318 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.050444 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:33.054687 (+ 4243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:33.054693 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:33.054697 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:33.054756 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:33.057747 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:33.058522 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:33.058528 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:33.058533 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:33.058686 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:33.059178 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:33.059183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:33.059186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:33.059544 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:33.059680 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:33.062511 (+ 2831us) server_negotiation.cc:299] Negotiation successful
0810 06:40:33.062730 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":955,"thread_start_us":1601,"threads_started":1}
I20260810 06:40:33.063414 22004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:33.039981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40907 (local address 127.31.59.194:42331)
0810 06:40:33.041442 (+ 1461us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:33.041494 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:33.043690 (+ 2196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:33.044399 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:33.044412 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:33.045076 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:33.045890 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:33.045918 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.047779 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.047785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:33.049544 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:33.049568 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.050748 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.050755 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:33.051028 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:33.051794 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:33.051829 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:33.054422 (+ 2593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:33.057883 (+ 3461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:33.057893 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:33.057911 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:33.058352 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:33.058814 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:33.058819 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:33.058822 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:33.059049 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:33.059758 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:33.059766 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:33.060187 (+ 421us) client_negotiation.cc:769] Sending connection context
0810 06:40:33.060463 (+ 276us) client_negotiation.cc:240] Negotiation successful
0810 06:40:33.060994 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1182,"thread_start_us":956,"threads_started":1}
I20260810 06:40:33.065351 22000 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40907
I20260810 06:40:33.065667 22000 heartbeater.cc:464] Registering TS with master...
I20260810 06:40:33.066604 22000 heartbeater.cc:511] Master 127.31.59.254:40907 requested a full tablet report, sending...
I20260810 06:40:33.068706 21500 ts_manager.cc:194] Registered new tserver with Master: da5a85c35f0c4c54949b96adb1ecc4c3 (127.31.59.194:35525)
I20260810 06:40:33.070991 21500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:42331
Aug 10 06:40:33 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344032, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:40:33.413715 21795 heartbeater.cc:464] Registering TS with master...
I20260810 06:40:33.414145 21795 heartbeater.cc:511] Master 127.31.59.254:40907 requested a full tablet report, sending...
I20260810 06:40:33.415455 21500 ts_manager.cc:194] Registered new tserver with Master: 0944caa46362487fb58b91ec2aa2d8f9 (127.31.59.193:34459)
I20260810 06:40:33.416698 21500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:52697
W20260810 06:40:33.492262 22006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:33.492534 22006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:33.492676 22006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:33.499843 22006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:33.499955 22006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:33.500115 22006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:40:33.509791 22006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.59.212:35477/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:43537
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=38903
--tserver_master_addrs=127.31.59.254:40907
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:33.511118 22006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:33.512398 22006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:33.534862 22014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:33.534938 22017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:33.536060 22006 server_base.cc:1034] running on GCE node
W20260810 06:40:33.535899 22019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:33.536847 22006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:33.538668 22006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:33.539927 22006 hybrid_clock.cc:648] HybridClock initialized: now 1786344033539852 us; error 59 us; skew 500 ppm
I20260810 06:40:33.540263 22006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:33 dist-test-slave-4tdj krb5kdc[19891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344033, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:33.543358 22006 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:40:33.550050 22006 webserver.cc:460] Webserver started at http://127.31.59.195:38903/ using document root <none> and password file <none>
I20260810 06:40:33.550793 22006 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:33.550932 22006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:40:33 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344033, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:40:33.569366 22006 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 06:40:33.574230 22029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:33.576090 22006 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260810 06:40:33.576282 22006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "f710875ed20742ed83f2039fc4189030"
format_stamp: "Formatted at 2026-08-10 06:40:30 on dist-test-slave-4tdj"
server_key: "80de8bb840b17979c776505b33f20dda"
server_key_iv: "eadad6279053954b6967642ac2894ca1"
server_key_version: "kuduclusterkey@0"
I20260810 06:40:33.577330 22006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:33.608330 22006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:33.612385 22006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:33.612656 22006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:33.613664 22006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:40:33.615272 22006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:40:33.615377 22006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:33.615478 22006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:40:33.615562 22006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:40:33 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344032, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for HTTP/127.31.59.212@KRBTEST.COM
I20260810 06:40:33.787288 22006 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:43537
I20260810 06:40:33.787357 22193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:43537 every 8 connection(s)
I20260810 06:40:33.788736 22006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:40:33.794483 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 22006
Aug 10 06:40:33 dist-test-slave-4tdj krb5kdc[19891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344033, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:40:33.856320 22197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:33.795322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:51875 (local address 127.31.59.254:40907)
0810 06:40:33.824708 (+ 29386us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:33.824717 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:33.829124 (+ 4407us) server_negotiation.cc:407] Connection header received
0810 06:40:33.829206 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:33.829213 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:33.829327 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:33.829434 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:33.836034 (+ 6600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.837205 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.840243 (+ 3038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.840895 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.840993 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:33.845528 (+ 4535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:33.845535 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:33.845540 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:33.845578 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:33.849036 (+ 3458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:33.849990 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:33.849996 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:33.849999 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:33.850147 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:33.850721 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:33.850726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:33.850729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:33.851897 (+ 1168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:33.852042 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:33.855595 (+ 3553us) server_negotiation.cc:299] Negotiation successful
0810 06:40:33.855821 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29219,"thread_start_us":29038,"threads_started":1}
I20260810 06:40:33.858982 22196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:33.793530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40907 (local address 127.31.59.195:51875)
0810 06:40:33.825811 (+ 32281us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:33.825881 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:33.828698 (+ 2817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:33.833411 (+ 4713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:33.833426 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:33.834172 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:33.835145 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:33.835173 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.837399 (+ 2226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.837404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:33.839968 (+ 2564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:33.839993 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.841328 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.841334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:33.841640 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:33.842432 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:33.842462 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:33.845239 (+ 2777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:33.849203 (+ 3964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:33.849213 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:33.849232 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:33.849849 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:33.850317 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:33.850322 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:33.850325 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:33.850585 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:33.853005 (+ 2420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:33.853013 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:33.855138 (+ 2125us) client_negotiation.cc:769] Sending connection context
0810 06:40:33.856885 (+ 1747us) client_negotiation.cc:240] Negotiation successful
0810 06:40:33.857480 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31877,"thread_start_us":31622,"threads_started":1}
I20260810 06:40:33.859064 22201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:33.795763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44550 (local address 127.31.59.254:40907)
0810 06:40:33.825082 (+ 29319us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:33.825088 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:33.830202 (+ 5114us) server_negotiation.cc:407] Connection header received
0810 06:40:33.830516 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:33.830521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:33.830602 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:33.830680 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:33.835999 (+ 5319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.837202 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.839222 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.839862 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.839981 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:33.850912 (+ 10931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:33.850918 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:33.850923 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:33.850972 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:33.854622 (+ 3650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:33.856018 (+ 1396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:33.856024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:33.856027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:33.856185 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:33.856650 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:33.856655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:33.856659 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:33.857004 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:33.857264 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:33.858469 (+ 1205us) server_negotiation.cc:299] Negotiation successful
0810 06:40:33.858700 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29222,"thread_start_us":29108,"threads_started":1}
I20260810 06:40:33.861138 22194 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40907
I20260810 06:40:33.861452 22194 heartbeater.cc:464] Registering TS with master...
I20260810 06:40:33.862349 22194 heartbeater.cc:511] Master 127.31.59.254:40907 requested a full tablet report, sending...
I20260810 06:40:33.864003 21500 ts_manager.cc:194] Registered new tserver with Master: f710875ed20742ed83f2039fc4189030 (127.31.59.195:43537)
I20260810 06:40:33.865074 21500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:51875
I20260810 06:40:33.874542 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:40:33.906924 22201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:33.885366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44560 (local address 127.31.59.254:40907)
0810 06:40:33.885561 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:33.885568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:33.894831 (+ 9263us) server_negotiation.cc:407] Connection header received
0810 06:40:33.895008 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:33.895014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:33.895085 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:33.895166 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:33.896735 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.897535 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.898913 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.899307 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.899413 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:33.901881 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:33.901885 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:33.901889 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:33.901922 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:33.904385 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:33.905017 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:33.905023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:33.905027 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:33.905147 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:33.905505 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:33.905511 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:33.905515 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:33.905828 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:33.906045 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:33.906394 (+ 349us) server_negotiation.cc:299] Negotiation successful
0810 06:40:33.906621 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:40:33.911947 21500 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:40:33.920370 21500 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:40:33.977435 22227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:33.945534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59260 (local address 127.31.59.193:34459)
0810 06:40:33.967764 (+ 22230us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:33.967775 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:33.967808 (+ 33us) server_negotiation.cc:407] Connection header received
0810 06:40:33.967916 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:33.967926 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:33.968220 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:33.970323 (+ 2103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:40:33.970580 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.972035 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.974836 (+ 2801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.976291 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.976436 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:33.976686 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:33.976936 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 06:40:33.977110 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22055,"thread_start_us":21890,"threads_started":1}
I20260810 06:40:33.977506 22226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:33.945036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34459 (local address 127.0.0.1:59260)
0810 06:40:33.958173 (+ 13137us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:33.958212 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:33.958422 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:33.969644 (+ 11222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:33.969649 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:33.969677 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:40:33.970067 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:33.970089 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.972208 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.972213 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:33.974663 (+ 2450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:33.974679 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.976500 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.976507 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:33.976683 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:33.976735 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 06:40:33.976881 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 06:40:33.977052 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12854,"mutex_wait_us":17,"thread_start_us":13582,"threads_started":1}
I20260810 06:40:33.978605 22228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:33.945824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:43537 (local address 127.0.0.1:32772)
0810 06:40:33.962152 (+ 16328us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:33.962171 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:33.962310 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:33.969333 (+ 7023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:33.969340 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:33.969371 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:40:33.969829 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:33.969841 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.971921 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.971926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:33.975467 (+ 3541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:33.975481 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.977465 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.977472 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:33.977920 (+ 448us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:33.977952 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:40:33.978033 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:40:33.978159 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16193,"mutex_wait_us":27,"thread_start_us":16559,"threads_started":1}
I20260810 06:40:33.979313 22229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:33.946238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32772 (local address 127.31.59.195:43537)
0810 06:40:33.968846 (+ 22608us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:33.968851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:33.968878 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:40:33.968948 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:33.968954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:33.969137 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:33.969303 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:40:33.970059 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.971766 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.975651 (+ 3885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.976990 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.977100 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:33.977303 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:33.978828 (+ 1525us) server_negotiation.cc:299] Negotiation successful
0810 06:40:33.978987 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22452,"thread_start_us":22317,"threads_started":1}
I20260810 06:40:33.981604 21694 tablet_service.cc:1467] Processing CreateTablet for tablet 2e8fba6db43942e99a01fe20e5acd899 (DEFAULT_TABLE table=test-table [id=a589d3310d1e480d87b26228975f8d8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:40:33.982656 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:33.959287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:35525 (local address 127.0.0.1:56232)
0810 06:40:33.974741 (+ 15454us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:33.974758 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:33.974878 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:33.975668 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:33.975672 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:33.975698 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:40:33.976028 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:33.976039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.978061 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.978066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:33.980167 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:33.980181 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.981687 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.981693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:33.981855 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:33.981907 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 06:40:33.981993 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 06:40:33.982416 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15346,"thread_start_us":15025,"threads_started":1}
I20260810 06:40:33.983211 22231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:33.959539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56232 (local address 127.31.59.194:35525)
0810 06:40:33.973953 (+ 14414us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:33.973961 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:33.974997 (+ 1036us) server_negotiation.cc:407] Connection header received
0810 06:40:33.975105 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:33.975112 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:33.975335 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:33.975512 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:40:33.976216 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.977928 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.980353 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:33.981475 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:33.981598 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:33.981780 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:33.982180 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 06:40:33.982940 (+ 760us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14255,"thread_start_us":13701,"threads_started":1}
I20260810 06:40:33.982906 22099 tablet_service.cc:1467] Processing CreateTablet for tablet 2e8fba6db43942e99a01fe20e5acd899 (DEFAULT_TABLE table=test-table [id=a589d3310d1e480d87b26228975f8d8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:40:33.983631 21694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e8fba6db43942e99a01fe20e5acd899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:33.984277 22099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e8fba6db43942e99a01fe20e5acd899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:33.988790 21911 tablet_service.cc:1467] Processing CreateTablet for tablet 2e8fba6db43942e99a01fe20e5acd899 (DEFAULT_TABLE table=test-table [id=a589d3310d1e480d87b26228975f8d8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:40:33.989787 21911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e8fba6db43942e99a01fe20e5acd899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:33.999032 22235 tablet_bootstrap.cc:492] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9: Bootstrap starting.
I20260810 06:40:33.999835 22236 tablet_bootstrap.cc:492] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030: Bootstrap starting.
I20260810 06:40:34.001600 22238 tablet_bootstrap.cc:492] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3: Bootstrap starting.
I20260810 06:40:34.003486 22236 tablet_bootstrap.cc:654] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:34.004372 22238 tablet_bootstrap.cc:654] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:34.004796 22236 log.cc:824] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:34.005306 22235 tablet_bootstrap.cc:654] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:34.005396 22238 log.cc:824] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:34.006572 22235 log.cc:824] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:34.009714 22238 tablet_bootstrap.cc:492] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3: No bootstrap required, opened a new log
I20260810 06:40:34.009714 22235 tablet_bootstrap.cc:492] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9: No bootstrap required, opened a new log
I20260810 06:40:34.009991 22235 ts_tablet_manager.cc:1397] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260810 06:40:34.009991 22238 ts_tablet_manager.cc:1397] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 06:40:34.010390 22236 tablet_bootstrap.cc:492] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030: No bootstrap required, opened a new log
I20260810 06:40:34.010639 22236 ts_tablet_manager.cc:1397] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260810 06:40:34.019193 22236 raft_consensus.cc:348] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0944caa46362487fb58b91ec2aa2d8f9" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34459 } } peers { permanent_uuid: "da5a85c35f0c4c54949b96adb1ecc4c3" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35525 } } peers { permanent_uuid: "f710875ed20742ed83f2039fc4189030" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43537 } }
I20260810 06:40:34.019615 22236 raft_consensus.cc:374] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:34.019716 22236 raft_consensus.cc:729] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f710875ed20742ed83f2039fc4189030, State: Initialized, Role: FOLLOWER
I20260810 06:40:34.020316 22236 consensus_queue.cc:260] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0944caa46362487fb58b91ec2aa2d8f9" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34459 } } peers { permanent_uuid: "da5a85c35f0c4c54949b96adb1ecc4c3" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35525 } } peers { permanent_uuid: "f710875ed20742ed83f2039fc4189030" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43537 } }
I20260810 06:40:34.020628 22235 raft_consensus.cc:348] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0944caa46362487fb58b91ec2aa2d8f9" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34459 } } peers { permanent_uuid: "da5a85c35f0c4c54949b96adb1ecc4c3" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35525 } } peers { permanent_uuid: "f710875ed20742ed83f2039fc4189030" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43537 } }
I20260810 06:40:34.021149 22235 raft_consensus.cc:374] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:34.021013 22238 raft_consensus.cc:348] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0944caa46362487fb58b91ec2aa2d8f9" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34459 } } peers { permanent_uuid: "da5a85c35f0c4c54949b96adb1ecc4c3" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35525 } } peers { permanent_uuid: "f710875ed20742ed83f2039fc4189030" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43537 } }
I20260810 06:40:34.021294 22235 raft_consensus.cc:729] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0944caa46362487fb58b91ec2aa2d8f9, State: Initialized, Role: FOLLOWER
I20260810 06:40:34.021377 22194 heartbeater.cc:503] Master 127.31.59.254:40907 was elected leader, sending a full tablet report...
I20260810 06:40:34.021487 22238 raft_consensus.cc:374] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:34.021638 22238 raft_consensus.cc:729] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da5a85c35f0c4c54949b96adb1ecc4c3, State: Initialized, Role: FOLLOWER
I20260810 06:40:34.021934 22235 consensus_queue.cc:260] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0944caa46362487fb58b91ec2aa2d8f9" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34459 } } peers { permanent_uuid: "da5a85c35f0c4c54949b96adb1ecc4c3" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35525 } } peers { permanent_uuid: "f710875ed20742ed83f2039fc4189030" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43537 } }
I20260810 06:40:34.022285 22238 consensus_queue.cc:260] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0944caa46362487fb58b91ec2aa2d8f9" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34459 } } peers { permanent_uuid: "da5a85c35f0c4c54949b96adb1ecc4c3" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35525 } } peers { permanent_uuid: "f710875ed20742ed83f2039fc4189030" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43537 } }
I20260810 06:40:34.023162 21795 heartbeater.cc:503] Master 127.31.59.254:40907 was elected leader, sending a full tablet report...
I20260810 06:40:34.024039 22235 ts_tablet_manager.cc:1428] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260810 06:40:34.026813 22236 ts_tablet_manager.cc:1428] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030: Time spent starting tablet: real 0.016s user 0.011s sys 0.000s
I20260810 06:40:34.027726 22000 heartbeater.cc:503] Master 127.31.59.254:40907 was elected leader, sending a full tablet report...
I20260810 06:40:34.028283 22238 ts_tablet_manager.cc:1428] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3: Time spent starting tablet: real 0.018s user 0.014s sys 0.004s
W20260810 06:40:34.043326 22002 tablet.cc:2369] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:40:34.075580 22244 raft_consensus.cc:482] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:40:34.075861 22244 raft_consensus.cc:504] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0944caa46362487fb58b91ec2aa2d8f9" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34459 } } peers { permanent_uuid: "da5a85c35f0c4c54949b96adb1ecc4c3" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35525 } } peers { permanent_uuid: "f710875ed20742ed83f2039fc4189030" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43537 } }
I20260810 06:40:34.077416 22244 leader_election.cc:288] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0944caa46362487fb58b91ec2aa2d8f9 (127.31.59.193:34459), da5a85c35f0c4c54949b96adb1ecc4c3 (127.31.59.194:35525)
I20260810 06:40:34.083576 22196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:34.077656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34459 (local address 127.31.59.195:32775)
0810 06:40:34.077812 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:34.077832 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:34.077961 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:34.078264 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:34.078268 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:34.078320 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:40:34.078500 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:34.078512 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:34.079761 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:34.079765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:34.081732 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:34.081749 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:34.082995 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:34.083000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:34.083133 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:34.083159 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 06:40:34.083242 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 06:40:34.083344 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":27}
I20260810 06:40:34.083637 22227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:34.077777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:32775 (local address 127.31.59.193:34459)
0810 06:40:34.077975 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:34.077981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:34.078001 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:40:34.078080 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:34.078085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:34.078154 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:34.078228 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:40:34.078677 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:34.079586 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:34.081888 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:34.082829 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:34.082994 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:34.083077 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:34.083275 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 06:40:34.083398 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:40:34.084578 22231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:34.078447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:35853 (local address 127.31.59.194:35525)
0810 06:40:34.078664 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:34.078670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:34.078688 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:40:34.078793 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:34.078798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:34.078865 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:34.078953 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:40:34.079383 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:34.080201 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:34.082604 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:34.083740 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:34.083881 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:34.083944 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:34.084219 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 06:40:34.084333 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:40:34.084652 22247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:34.078023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:35525 (local address 127.31.59.195:35853)
0810 06:40:34.078575 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:34.078593 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:34.078697 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:34.078956 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:34.078959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:34.078975 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:40:34.079259 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:34.079267 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:34.080369 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:34.080375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:34.082437 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:34.082450 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:34.083920 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:34.083925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:34.084079 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:34.084101 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 06:40:34.084178 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 06:40:34.084301 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":1,"thread_start_us":479,"threads_started":1}
I20260810 06:40:34.084630 21723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e8fba6db43942e99a01fe20e5acd899" candidate_uuid: "f710875ed20742ed83f2039fc4189030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0944caa46362487fb58b91ec2aa2d8f9" is_pre_election: true
I20260810 06:40:34.085268 21723 raft_consensus.cc:2454] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f710875ed20742ed83f2039fc4189030 in term 0.
I20260810 06:40:34.085897 22033 leader_election.cc:302] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0944caa46362487fb58b91ec2aa2d8f9, f710875ed20742ed83f2039fc4189030; no voters:
W20260810 06:40:34.086256 22195 tablet.cc:2369] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:40:34.086356 22244 raft_consensus.cc:2781] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:40:34.086498 22244 raft_consensus.cc:482] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:40:34.086611 22244 raft_consensus.cc:3037] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:34.086503 21936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e8fba6db43942e99a01fe20e5acd899" candidate_uuid: "f710875ed20742ed83f2039fc4189030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da5a85c35f0c4c54949b96adb1ecc4c3" is_pre_election: true
I20260810 06:40:34.086934 21936 raft_consensus.cc:2454] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f710875ed20742ed83f2039fc4189030 in term 0.
I20260810 06:40:34.089092 22244 raft_consensus.cc:504] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0944caa46362487fb58b91ec2aa2d8f9" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34459 } } peers { permanent_uuid: "da5a85c35f0c4c54949b96adb1ecc4c3" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35525 } } peers { permanent_uuid: "f710875ed20742ed83f2039fc4189030" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43537 } }
I20260810 06:40:34.089689 22244 leader_election.cc:288] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [CANDIDATE]: Term 1 election: Requested vote from peers 0944caa46362487fb58b91ec2aa2d8f9 (127.31.59.193:34459), da5a85c35f0c4c54949b96adb1ecc4c3 (127.31.59.194:35525)
I20260810 06:40:34.090073 21723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e8fba6db43942e99a01fe20e5acd899" candidate_uuid: "f710875ed20742ed83f2039fc4189030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0944caa46362487fb58b91ec2aa2d8f9"
I20260810 06:40:34.090098 21936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e8fba6db43942e99a01fe20e5acd899" candidate_uuid: "f710875ed20742ed83f2039fc4189030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da5a85c35f0c4c54949b96adb1ecc4c3"
I20260810 06:40:34.090250 21936 raft_consensus.cc:3037] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:34.090258 21723 raft_consensus.cc:3037] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:34.092299 21936 raft_consensus.cc:2454] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f710875ed20742ed83f2039fc4189030 in term 1.
I20260810 06:40:34.092639 22033 leader_election.cc:302] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da5a85c35f0c4c54949b96adb1ecc4c3, f710875ed20742ed83f2039fc4189030; no voters:
I20260810 06:40:34.092646 21723 raft_consensus.cc:2454] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f710875ed20742ed83f2039fc4189030 in term 1.
I20260810 06:40:34.092917 22244 raft_consensus.cc:2781] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:40:34.093367 22244 raft_consensus.cc:686] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [term 1 LEADER]: Becoming Leader. State: Replica: f710875ed20742ed83f2039fc4189030, State: Running, Role: LEADER
I20260810 06:40:34.093827 22244 consensus_queue.cc:237] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0944caa46362487fb58b91ec2aa2d8f9" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34459 } } peers { permanent_uuid: "da5a85c35f0c4c54949b96adb1ecc4c3" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35525 } } peers { permanent_uuid: "f710875ed20742ed83f2039fc4189030" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43537 } }
I20260810 06:40:34.098652 21499 catalog_manager.cc:5495] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 reported cstate change: term changed from 0 to 1, leader changed from <none> to f710875ed20742ed83f2039fc4189030 (127.31.59.195). New cstate: current_term: 1 leader_uuid: "f710875ed20742ed83f2039fc4189030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0944caa46362487fb58b91ec2aa2d8f9" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da5a85c35f0c4c54949b96adb1ecc4c3" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f710875ed20742ed83f2039fc4189030" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43537 } health_report { overall_health: HEALTHY } } }
W20260810 06:40:34.128541 21797 tablet.cc:2369] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:40:34.146596 22229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:34.141961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32780 (local address 127.31.59.195:43537)
0810 06:40:34.142144 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:34.142148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:34.142165 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:40:34.142327 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:34.142333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:34.142420 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:34.142571 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:40:34.142866 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:34.143562 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:34.144819 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:34.145190 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:34.145290 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:34.145673 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:40:34.146138 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:40:34.146252 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:34.146287 (+ 35us) server_negotiation.cc:299] Negotiation successful
0810 06:40:34.146373 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:40:34.154512 21936 raft_consensus.cc:1261] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3 [term 1 FOLLOWER]: Refusing update from remote peer f710875ed20742ed83f2039fc4189030: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:40:34.154528 21723 raft_consensus.cc:1261] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9 [term 1 FOLLOWER]: Refusing update from remote peer f710875ed20742ed83f2039fc4189030: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:40:34.155408 22248 consensus_queue.cc:1035] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0944caa46362487fb58b91ec2aa2d8f9" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:40:34.155747 22244 consensus_queue.cc:1035] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da5a85c35f0c4c54949b96adb1ecc4c3" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:40:34.170351 22254 mvcc.cc:205] Tried to move back new op lower bound from 7316865163887464448 to 7316865163652354048. Current Snapshot: MvccSnapshot[applied={T|T < 7316865163887464448}]
I20260810 06:40:34.187783 21500 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:40:34.188058 21500 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:40:34.191395 21500 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4c8a90d6914048b9b95f308993c1b450: Sending DeleteTablet for 3 replicas of tablet 2e8fba6db43942e99a01fe20e5acd899
I20260810 06:40:34.192572 21911 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e8fba6db43942e99a01fe20e5acd899 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:40:34 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:56232
I20260810 06:40:34.192557 21694 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e8fba6db43942e99a01fe20e5acd899 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:40:34 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:59260
I20260810 06:40:34.192909 22099 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e8fba6db43942e99a01fe20e5acd899 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:40:34 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:32772
I20260810 06:40:34.199476 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 21552
I20260810 06:40:34.203687 22268 tablet_replica.cc:330] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3: stopping tablet replica
I20260810 06:40:34.204260 22268 raft_consensus.cc:2229] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:40:34.204924 22268 raft_consensus.cc:2258] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:40:34.205955 22267 tablet_replica.cc:330] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9: stopping tablet replica
I20260810 06:40:34.207688 22269 tablet_replica.cc:330] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030: stopping tablet replica
I20260810 06:40:34.207887 22268 ts_tablet_manager.cc:1905] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:40:34.208894 22267 raft_consensus.cc:2229] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:40:34.209411 22267 raft_consensus.cc:2258] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:40:34.209465 22269 raft_consensus.cc:2229] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:40:34.210242 22269 raft_consensus.cc:2258] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:40:34.211741 22267 ts_tablet_manager.cc:1905] T 2e8fba6db43942e99a01fe20e5acd899 P 0944caa46362487fb58b91ec2aa2d8f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:40:34.214545 22269 ts_tablet_manager.cc:1905] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:40:34.216643 22268 ts_tablet_manager.cc:1918] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:40:34.216864 22268 log.cc:1196] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/2e8fba6db43942e99a01fe20e5acd899
I20260810 06:40:34.217547 22268 ts_tablet_manager.cc:1939] T 2e8fba6db43942e99a01fe20e5acd899 P da5a85c35f0c4c54949b96adb1ecc4c3: Deleting consensus metadata
I20260810 06:40:34.218838 22227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:34.201385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59270 (local address 127.31.59.193:34459)
0810 06:40:34.201568 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:34.201574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:34.201594 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:40:34.201658 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:34.201665 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:34.201757 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:34.201869 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:34.204946 (+ 3077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:34.206077 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:34.207956 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:34.208549 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:34.208683 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:34.212305 (+ 3622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:34.212317 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:34.212334 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:34.212372 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:34.215907 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:34.216680 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:34.216689 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:34.216693 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:34.216847 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:34.217254 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:34.217259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:34.217263 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:34.217696 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:34.217993 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:34.218261 (+ 268us) server_negotiation.cc:299] Negotiation successful
0810 06:40:34.218488 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:40:34.219515 21470 catalog_manager.cc:4841] TS da5a85c35f0c4c54949b96adb1ecc4c3 (127.31.59.194:35525): tablet 2e8fba6db43942e99a01fe20e5acd899 (table test-table [id=a589d3310d1e480d87b26228975f8d8d]) successfully deleted
I20260810 06:40:34.219664 21790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:34.224980 22269 ts_tablet_manager.cc:1918] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:40:34.225142 22269 log.cc:1196] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/2e8fba6db43942e99a01fe20e5acd899
I20260810 06:40:34.225602 22269 ts_tablet_manager.cc:1939] T 2e8fba6db43942e99a01fe20e5acd899 P f710875ed20742ed83f2039fc4189030: Deleting consensus metadata
I20260810 06:40:34.227052 21465 catalog_manager.cc:4841] TS f710875ed20742ed83f2039fc4189030 (127.31.59.195:43537): tablet 2e8fba6db43942e99a01fe20e5acd899 (table test-table [id=a589d3310d1e480d87b26228975f8d8d]) successfully deleted
I20260810 06:40:34.327399 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 21552
I20260810 06:40:34.348990 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 21802
W20260810 06:40:34.349054 21470 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.59.193:34459 (error 108)
I20260810 06:40:34.353317 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:34.352819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34459 (local address 127.0.0.1:59284)
0810 06:40:34.353022 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:34.353146 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:34459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260810 06:40:34.353600 21470 catalog_manager.cc:4584] TS 0944caa46362487fb58b91ec2aa2d8f9 (127.31.59.193:34459): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e8fba6db43942e99a01fe20e5acd899: Network error: Client connection negotiation failed: client connection to 127.31.59.193:34459: connect: Connection refused (error 111)
I20260810 06:40:34.363564 22231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:34.349746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56240 (local address 127.31.59.194:35525)
0810 06:40:34.349954 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:34.349972 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:34.350260 (+ 288us) server_negotiation.cc:407] Connection header received
0810 06:40:34.350355 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:34.350360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:34.350442 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:34.350549 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:34.352596 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:34.353543 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:34.355017 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:34.355394 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:34.355504 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:34.357938 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:34.357952 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:34.357966 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:34.358010 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:34.360783 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:34.361460 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:34.361467 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:34.361470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:34.361607 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:34.361973 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:34.361977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:34.361985 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:34.362392 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:34.362690 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:34.363001 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 06:40:34.363234 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:40:34.364300 21994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:34.407380 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:34.406813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34459 (local address 127.0.0.1:59298)
0810 06:40:34.407055 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:34.407188 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:34459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 06:40:34.408618 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:34.408150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34459 (local address 127.0.0.1:59300)
0810 06:40:34.408363 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:34.408452 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:34459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 06:40:34.453306 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 21802
I20260810 06:40:34.469233 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:34.468761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34459 (local address 127.0.0.1:59304)
0810 06:40:34.468957 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:34.469045 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:34459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 06:40:34.470350 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:34.469967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34459 (local address 127.0.0.1:59314)
0810 06:40:34.470114 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:34.470188 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:34459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 06:40:34.480829 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 22006
I20260810 06:40:34.494437 22229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:34.481566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32790 (local address 127.31.59.195:43537)
0810 06:40:34.481755 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:34.481761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:34.481781 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:40:34.481940 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:34.481945 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:34.482031 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:34.482125 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:34.483480 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:34.484270 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:34.485578 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:34.486027 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:34.486137 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:34.488661 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:34.488676 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:34.488691 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:34.488730 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:34.491707 (+ 2977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:34.492365 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:34.492372 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:34.492374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:34.492539 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:34.492889 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:34.492893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:34.492895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:34.493261 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:34.493576 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:34.493863 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 06:40:34.494082 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 06:40:34.495083 22187 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:34.550952 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:34.550426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34459 (local address 127.0.0.1:59326)
0810 06:40:34.550646 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:34.550761 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:34459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 06:40:34.552045 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:34.551616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34459 (local address 127.0.0.1:59340)
0810 06:40:34.551806 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:34.551894 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:34459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260810 06:40:34.612510 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 22006
I20260810 06:40:34.638128 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 21437
I20260810 06:40:34.639303 21545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:34.749806 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:34.749071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34459 (local address 127.0.0.1:59350)
0810 06:40:34.749429 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:34.749597 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:34459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212}
I20260810 06:40:34.750278 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 21437
I20260810 06:40:34.781670 31983 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 06:40:35.126478 31983 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 06:40:35.126713 31983 mini_ranger.cc:66] Stopping Ranger...
I20260810 06:40:35.471414 31983 mini_ranger.cc:68] Stopped Ranger
2026-08-10 06:40:35.471 UTC [19944] LOG: received smart shutdown request
2026-08-10 06:40:35.474 UTC [19944] LOG: background worker "logical replication launcher" (PID 19956) exited with exit code 1
2026-08-10 06:40:35.475 UTC [19949] LOG: shutting down
2026-08-10 06:40:35.613 UTC [19944] LOG: database system is shut down
2026-08-10T06:40:35Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51559 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:40:35 dist-test-slave-4tdj krb5kdc[22293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:40:35 dist-test-slave-4tdj krb5kdc[22293](info): set up 2 sockets
Aug 10 06:40:35 dist-test-slave-4tdj krb5kdc[22293](info): commencing operation
krb5kdc: starting...
W20260810 06:40:36.946864 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.006s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:40:37 dist-test-slave-4tdj krb5kdc[22293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:40:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:40:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:40:37.195636 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:38743
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:38743,127.31.59.253:36263,127.31.59.252:45009
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:37.519466 22312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:37.519750 22312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:37.519887 22312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:37.526839 22312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:37.526947 22312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:37.527024 22312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:37.527098 22312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:37.527169 22312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:37.536592 22312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:38743,127.31.59.253:36263,127.31.59.252:45009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:38743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:37.537817 22312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:37.539099 22312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:37.549939 22322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:37.550037 22326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:37.550316 22321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:37.550889 22312 server_base.cc:1034] running on GCE node
I20260810 06:40:37.551617 22312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:37.552860 22312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:37.554065 22312 hybrid_clock.cc:648] HybridClock initialized: now 1786344037554018 us; error 37 us; skew 500 ppm
I20260810 06:40:37.554425 22312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:37 dist-test-slave-4tdj krb5kdc[22293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344037, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:37.560222 22312 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:40:37.561592 22312 webserver.cc:460] Webserver started at http://127.31.59.254:34493/ using document root <none> and password file <none>
I20260810 06:40:37.562206 22312 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:37.562330 22312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:37.562611 22312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:37.565439 22312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "935dae40e80843748c24ef208194bd3a"
format_stamp: "Formatted at 2026-08-10 06:40:37 on dist-test-slave-4tdj"
server_key: "bdacc1153bfa1c4cd2c6abc6efd012d4"
server_key_iv: "67fd314fe76880a7c87da86c3189153b"
server_key_version: "encryptionkey@0"
I20260810 06:40:37.566184 22312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "935dae40e80843748c24ef208194bd3a"
format_stamp: "Formatted at 2026-08-10 06:40:37 on dist-test-slave-4tdj"
server_key: "bdacc1153bfa1c4cd2c6abc6efd012d4"
server_key_iv: "67fd314fe76880a7c87da86c3189153b"
server_key_version: "encryptionkey@0"
I20260810 06:40:37.570978 22312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 06:40:37.574220 22337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:37.575459 22312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260810 06:40:37.575614 22312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "935dae40e80843748c24ef208194bd3a"
format_stamp: "Formatted at 2026-08-10 06:40:37 on dist-test-slave-4tdj"
server_key: "bdacc1153bfa1c4cd2c6abc6efd012d4"
server_key_iv: "67fd314fe76880a7c87da86c3189153b"
server_key_version: "encryptionkey@0"
I20260810 06:40:37.575767 22312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:37.616151 22312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:37.620323 22312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:37.620572 22312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:37.637861 22312 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:38743
I20260810 06:40:37.637897 22411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:38743 every 8 connection(s)
I20260810 06:40:37.639189 22312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:40:37.642771 22412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:37.647514 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 22312
I20260810 06:40:37.647725 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:40:37.648356 31983 external_mini_cluster.cc:1428] Setting key 9786eb3f11d03666f8ec81ecc5fa38fe
I20260810 06:40:37.649434 22412 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:40:37 dist-test-slave-4tdj krb5kdc[22293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344037, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:40:37.671300 22417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:37.652394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48974 (local address 127.31.59.254:38743)
0810 06:40:37.654000 (+ 1606us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:37.654008 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:37.654060 (+ 52us) server_negotiation.cc:407] Connection header received
0810 06:40:37.655556 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:37.655561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:37.655765 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:37.655947 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:37.657180 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:37.658386 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:37.660180 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:37.660717 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:37.660849 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:37.664544 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:37.664552 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:37.664563 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:37.664591 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:37.667249 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:37.668036 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:37.668042 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:37.668044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:37.668184 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:37.668624 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:37.668630 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:37.668633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:37.669106 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:37.669265 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:37.669887 (+ 622us) server_negotiation.cc:299] Negotiation successful
0810 06:40:37.670216 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":1400,"spinlock_wait_cycles":15744,"thread_start_us":1168,"threads_started":1}
I20260810 06:40:37.671340 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:37.652393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:38743 (local address 127.0.0.1:48974)
0810 06:40:37.653196 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:37.653249 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:37.655217 (+ 1968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:37.655982 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:37.655988 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:37.656670 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:37.656984 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:37.657001 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:37.658528 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:37.658531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:37.660008 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:37.660023 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:37.660982 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:37.660990 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:37.661149 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:37.661752 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:37.661789 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:37.664322 (+ 2533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:37.667378 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:37.667390 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:37.667401 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:37.667914 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:37.668291 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:37.668295 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:37.668299 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:37.668500 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:37.669257 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:37.669264 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:37.669498 (+ 234us) client_negotiation.cc:769] Sending connection context
0810 06:40:37.669791 (+ 293us) client_negotiation.cc:240] Negotiation successful
0810 06:40:37.670391 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":138,"thread_start_us":334,"threads_started":1}
WARNING: no policy specified for kudu/127.31.59.253@KRBTEST.COM; defaulting to no policy
I20260810 06:40:37.673923 22412 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:40:37.674975 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:37.674625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36263 (local address 127.0.0.1:48180)
0810 06:40:37.674773 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:37.674873 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 06:40:37.675235 22343 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36263: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.253@KRBTEST.COM" created.
I20260810 06:40:37.676594 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:37.676253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36263 (local address 127.0.0.1:48182)
0810 06:40:37.676383 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:37.676450 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 06:40:37.676780 22412 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.253:36263: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36263: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.253.keytab.
Entry for principal kudu/127.31.59.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.253.keytab.
I20260810 06:40:37.702553 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.253:36263
--webserver_interface=127.31.59.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.253.keytab
--principal=kudu/127.31.59.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:38743,127.31.59.253:36263,127.31.59.252:45009
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:40:37.726173 22412 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } attempt: 1
I20260810 06:40:37.727511 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:37.726960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36263 (local address 127.0.0.1:48192)
0810 06:40:37.727171 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:37.727342 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 06:40:37.728595 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:37.728190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36263 (local address 127.0.0.1:48198)
0810 06:40:37.728340 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:37.728421 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 06:40:37.728848 22412 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.253:36263: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36263: connect: Connection refused (error 111)
I20260810 06:40:37.797108 22412 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } attempt: 2
I20260810 06:40:37.798350 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:37.797809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36263 (local address 127.0.0.1:59780)
0810 06:40:37.798065 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:37.798161 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 06:40:37.799343 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:37.798973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36263 (local address 127.0.0.1:59794)
0810 06:40:37.799135 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:37.799209 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 06:40:37.799532 22412 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.253:36263: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36263: connect: Connection refused (error 111)
I20260810 06:40:37.890776 22412 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } attempt: 3
I20260810 06:40:37.891860 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:37.891412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36263 (local address 127.0.0.1:59804)
0810 06:40:37.891618 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:37.891726 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 06:40:37.892820 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:37.892454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36263 (local address 127.0.0.1:59818)
0810 06:40:37.892614 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:37.892681 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260810 06:40:37.892993 22412 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.253:36263: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36263: connect: Connection refused (error 111)
W20260810 06:40:38.020253 22421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:38.020591 22421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:38.020744 22421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:38.028824 22421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:38.028939 22421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:38.029016 22421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:38.029096 22421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:38.029156 22421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:38.036223 22412 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } attempt: 4
I20260810 06:40:38.037336 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.036810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36263 (local address 127.0.0.1:59822)
0810 06:40:38.037115 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.037201 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260810 06:40:38.038407 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.037996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36263 (local address 127.0.0.1:59830)
0810 06:40:38.038145 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.038235 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 06:40:38.038626 22412 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.253:36263: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36263: connect: Connection refused (error 111)
I20260810 06:40:38.038992 22421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:38743,127.31.59.253:36263,127.31.59.252:45009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.253.keytab
--principal=kudu/127.31.59.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.253:36263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.59.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:38.040247 22421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:38.041498 22421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:38.052613 22427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:38.052805 22429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:38.052775 22431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:38.053659 22421 server_base.cc:1034] running on GCE node
I20260810 06:40:38.054481 22421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:38.055764 22421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:38.056959 22421 hybrid_clock.cc:648] HybridClock initialized: now 1786344038056902 us; error 46 us; skew 500 ppm
I20260810 06:40:38.057279 22421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:38 dist-test-slave-4tdj krb5kdc[22293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344038, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:38.060314 22421 init.cc:377] Logged in from keytab as kudu/127.31.59.253@KRBTEST.COM (short username kudu)
I20260810 06:40:38.061676 22421 webserver.cc:460] Webserver started at http://127.31.59.253:33921/ using document root <none> and password file <none>
I20260810 06:40:38.062371 22421 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:38.062476 22421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:38.062726 22421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:38.065591 22421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-1/data/instance:
uuid: "ee59c7e874514948994d0b6084c93624"
format_stamp: "Formatted at 2026-08-10 06:40:38 on dist-test-slave-4tdj"
server_key: "8adfabdb8fee228a635f5b42118212f9"
server_key_iv: "7e166b0cceda5276e3c8045f57cfeef6"
server_key_version: "encryptionkey@0"
I20260810 06:40:38.066386 22421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-1/wal/instance:
uuid: "ee59c7e874514948994d0b6084c93624"
format_stamp: "Formatted at 2026-08-10 06:40:38 on dist-test-slave-4tdj"
server_key: "8adfabdb8fee228a635f5b42118212f9"
server_key_iv: "7e166b0cceda5276e3c8045f57cfeef6"
server_key_version: "encryptionkey@0"
I20260810 06:40:38.071295 22421 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 06:40:38.074616 22440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:38.075870 22421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:40:38.076010 22421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-1/wal
uuid: "ee59c7e874514948994d0b6084c93624"
format_stamp: "Formatted at 2026-08-10 06:40:38 on dist-test-slave-4tdj"
server_key: "8adfabdb8fee228a635f5b42118212f9"
server_key_iv: "7e166b0cceda5276e3c8045f57cfeef6"
server_key_version: "encryptionkey@0"
I20260810 06:40:38.076180 22421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:38.090402 22421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:38.094352 22421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:38.094576 22421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:38.111714 22421 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.253:36263
I20260810 06:40:38.111775 22522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.253:36263 every 8 connection(s)
I20260810 06:40:38.113015 22421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-1/data/info.pb
I20260810 06:40:38.114037 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 22421
I20260810 06:40:38.114248 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-1/wal/instance
I20260810 06:40:38.114905 31983 external_mini_cluster.cc:1428] Setting key a0f581f1a5c408a0497571683ba838d3
I20260810 06:40:38.117690 22523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:38.126569 22523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.59.252@KRBTEST.COM; defaulting to no policy
Aug 10 06:40:38 dist-test-slave-4tdj krb5kdc[22293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344038, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.253@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.252@KRBTEST.COM" created.
I20260810 06:40:38.148881 22417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.129137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37162 (local address 127.31.59.254:38743)
0810 06:40:38.129361 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:38.129368 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:38.130640 (+ 1272us) server_negotiation.cc:407] Connection header received
0810 06:40:38.133049 (+ 2409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:38.133055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:38.133129 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:38.133230 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:38.135190 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.135886 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.138142 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.138613 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.138724 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.142736 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:38.142741 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:38.142744 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:38.142779 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:38.145730 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.146542 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.146549 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.146552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.146697 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.147102 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.147106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.147109 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.147442 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:38.147589 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:38.148352 (+ 763us) server_negotiation.cc:299] Negotiation successful
0810 06:40:38.148574 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:40:38.149794 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.129630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:38743 (local address 127.0.0.1:37162)
0810 06:40:38.130459 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.130517 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:38.132743 (+ 2226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:38.133466 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:38.133477 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:38.134231 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:38.135005 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:38.135027 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.136065 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.136070 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:38.137969 (+ 1899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:38.137995 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.138849 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.138857 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:38.139165 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.139911 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:38.139941 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:38.142490 (+ 2549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:38.145868 (+ 3378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:38.145875 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:38.145888 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:38.146400 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:38.146795 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:38.146798 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:38.146801 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:38.146987 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:38.147587 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:38.147593 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:38.147950 (+ 357us) client_negotiation.cc:769] Sending connection context
0810 06:40:38.148208 (+ 258us) client_negotiation.cc:240] Negotiation successful
0810 06:40:38.148600 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":332,"threads_started":1}
I20260810 06:40:38.151659 22523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:40:38 dist-test-slave-4tdj krb5kdc[22293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344038, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.253@KRBTEST.COM for kudu/127.31.59.253@KRBTEST.COM
I20260810 06:40:38.168488 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.152448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36263 (local address 127.0.0.1:59832)
0810 06:40:38.152643 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.152665 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:38.152790 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:38.153546 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:38.153550 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:38.153928 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:38.154075 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:38.154084 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.155455 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.155460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:38.156680 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:38.156694 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.157515 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.157520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:38.157721 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.158268 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:38.158342 (+ 74us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:38.160889 (+ 2547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:38.165527 (+ 4638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:38.165532 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:38.165536 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:38.165983 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:38.166490 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:38.166495 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:38.166499 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:38.166699 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:38.167528 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:38.167533 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:38.167693 (+ 160us) client_negotiation.cc:769] Sending connection context
0810 06:40:38.167836 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 06:40:38.168106 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 06:40:38.169647 22530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.152500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59832 (local address 127.31.59.253:36263)
0810 06:40:38.153083 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:38.153090 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:38.153120 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:40:38.153169 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:38.153177 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:38.153396 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:38.153532 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:38.154234 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.155323 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.156832 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.157362 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.157461 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.161292 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:38.161309 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:38.161326 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:38.161369 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:38.165344 (+ 3975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.166128 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.166138 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.166142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.166331 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.166824 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.166829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.166832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.167277 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:38.167462 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:38.169013 (+ 1551us) server_negotiation.cc:299] Negotiation successful
0810 06:40:38.169268 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":255,"threads_started":1}
I20260810 06:40:38.173940 22523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:40:38.175418 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.174821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39784)
0810 06:40:38.175024 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.175174 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 06:40:38.175738 22442 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
I20260810 06:40:38.177242 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.176927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39794)
0810 06:40:38.177047 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.177138 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 06:40:38.177459 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:45009: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.252.keytab.
Entry for principal kudu/127.31.59.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.252.keytab.
I20260810 06:40:38.181308 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.252:45009
--webserver_interface=127.31.59.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.252.keytab
--principal=kudu/127.31.59.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:38743,127.31.59.253:36263,127.31.59.252:45009
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:40:38.226810 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } attempt: 1
I20260810 06:40:38.228026 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.227570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39802)
0810 06:40:38.227760 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.227870 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 06:40:38.229017 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.228692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39810)
0810 06:40:38.228840 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.228907 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 06:40:38.229223 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:45009: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
I20260810 06:40:38.297442 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } attempt: 2
I20260810 06:40:38.298679 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.298169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39816)
0810 06:40:38.298416 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.298512 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260810 06:40:38.299592 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.299267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39822)
0810 06:40:38.299407 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.299475 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 06:40:38.299786 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:45009: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
I20260810 06:40:38.337888 22412 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } attempt: 5
Aug 10 06:40:38 dist-test-slave-4tdj krb5kdc[22293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344037, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for kudu/127.31.59.253@KRBTEST.COM
I20260810 06:40:38.351832 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.338755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36263 (local address 127.0.0.1:59834)
0810 06:40:38.338975 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.338996 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:38.339156 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:38.339507 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:38.339513 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:38.339917 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:38.340125 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:38.340136 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.341150 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.341156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:38.342009 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:38.342022 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.342823 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.342828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:38.343013 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.343595 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:38.343614 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:38.345629 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:38.349527 (+ 3898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:38.349530 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:38.349533 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:38.349878 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:38.350286 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:38.350313 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:38.350317 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:38.350451 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:38.351122 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:38.351127 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:38.351289 (+ 162us) client_negotiation.cc:769] Sending connection context
0810 06:40:38.351400 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 06:40:38.351592 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 06:40:38.352159 22530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.338792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59834 (local address 127.31.59.253:36263)
0810 06:40:38.338984 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:38.338990 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:38.339077 (+ 87us) server_negotiation.cc:407] Connection header received
0810 06:40:38.339286 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:38.339291 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:38.339371 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:38.339460 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:38.340300 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.341014 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.342169 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.342680 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.342799 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.345884 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:38.345892 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:38.345897 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:38.345944 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:38.349367 (+ 3423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.349997 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.350003 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.350007 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.350175 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.350557 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.350563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.350566 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.350962 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:38.351128 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:38.351553 (+ 425us) server_negotiation.cc:299] Negotiation successful
0810 06:40:38.351803 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 06:40:38.353104 22412 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:40:38.354321 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.353908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39832)
0810 06:40:38.354079 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.354153 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 06:40:38.355352 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.355049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39846)
0810 06:40:38.355194 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.355258 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 06:40:38.355573 22412 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:45009: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
I20260810 06:40:38.390992 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } attempt: 3
I20260810 06:40:38.392071 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.391599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39856)
0810 06:40:38.391809 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.391909 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260810 06:40:38.393077 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.392653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39870)
0810 06:40:38.392879 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.392954 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260810 06:40:38.393280 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:45009: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
I20260810 06:40:38.406773 22412 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } attempt: 1
I20260810 06:40:38.407754 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.407332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39882)
0810 06:40:38.407511 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.407604 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 06:40:38.408751 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.408423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39886)
0810 06:40:38.408574 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.408645 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 06:40:38.408944 22412 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:45009: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
I20260810 06:40:38.477172 22412 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } attempt: 2
I20260810 06:40:38.478431 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.477891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39896)
0810 06:40:38.478121 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.478217 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260810 06:40:38.479509 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.479117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39898)
0810 06:40:38.479286 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.479359 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 06:40:38.479748 22412 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:45009: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
W20260810 06:40:38.509806 22535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:38.510075 22535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:38.510219 22535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:38.517287 22535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:38.517395 22535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:38.517475 22535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:38.517554 22535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:38.517616 22535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:38.527140 22535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:38743,127.31.59.253:36263,127.31.59.252:45009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.252.keytab
--principal=kudu/127.31.59.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.252:45009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.59.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:38.528389 22535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:38.529634 22535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:40:38.536484 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } attempt: 4
I20260810 06:40:38.537616 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.537118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39908)
0810 06:40:38.537346 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.537439 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 06:40:38.538702 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.538356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39920)
0810 06:40:38.538516 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.538595 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 06:40:38.538915 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:45009: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
W20260810 06:40:38.541041 22547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:38.541103 22541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:38.541049 22542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:38.542004 22535 server_base.cc:1034] running on GCE node
I20260810 06:40:38.542716 22535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:38.543926 22535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:38.545136 22535 hybrid_clock.cc:648] HybridClock initialized: now 1786344038545095 us; error 36 us; skew 500 ppm
I20260810 06:40:38.545456 22535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:38 dist-test-slave-4tdj krb5kdc[22293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344038, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:38.548426 22535 init.cc:377] Logged in from keytab as kudu/127.31.59.252@KRBTEST.COM (short username kudu)
I20260810 06:40:38.549769 22535 webserver.cc:460] Webserver started at http://127.31.59.252:35409/ using document root <none> and password file <none>
I20260810 06:40:38.550432 22535 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:38.550554 22535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:38.550830 22535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:38.553576 22535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-2/data/instance:
uuid: "3a73e909ba184c8cb8f7951802b60e87"
format_stamp: "Formatted at 2026-08-10 06:40:38 on dist-test-slave-4tdj"
server_key: "f6fdfe8d42a0a9ae208500da1f1cd682"
server_key_iv: "2e0226ac6812ffec233f5f8e4b65a7d3"
server_key_version: "encryptionkey@0"
I20260810 06:40:38.554380 22535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-2/wal/instance:
uuid: "3a73e909ba184c8cb8f7951802b60e87"
format_stamp: "Formatted at 2026-08-10 06:40:38 on dist-test-slave-4tdj"
server_key: "f6fdfe8d42a0a9ae208500da1f1cd682"
server_key_iv: "2e0226ac6812ffec233f5f8e4b65a7d3"
server_key_version: "encryptionkey@0"
I20260810 06:40:38.559269 22535 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 06:40:38.562566 22558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:38.563833 22535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:40:38.564020 22535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-2/wal
uuid: "3a73e909ba184c8cb8f7951802b60e87"
format_stamp: "Formatted at 2026-08-10 06:40:38 on dist-test-slave-4tdj"
server_key: "f6fdfe8d42a0a9ae208500da1f1cd682"
server_key_iv: "2e0226ac6812ffec233f5f8e4b65a7d3"
server_key_version: "encryptionkey@0"
I20260810 06:40:38.564215 22535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:38.585987 22412 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } attempt: 3
I20260810 06:40:38.587033 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.586576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39934)
0810 06:40:38.586787 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.586869 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 06:40:38.588140 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.587638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39942)
0810 06:40:38.587875 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.587983 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260810 06:40:38.588411 22412 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:45009: Network error: Client connection negotiation failed: client connection to 127.31.59.252:45009: connect: Connection refused (error 111)
I20260810 06:40:38.590845 22535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:38.594784 22535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:38.595060 22535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:38.612986 22535 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.252:45009
I20260810 06:40:38.612991 22633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.252:45009 every 8 connection(s)
I20260810 06:40:38.614476 22535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-2/data/info.pb
I20260810 06:40:38.615217 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 22535
I20260810 06:40:38.615408 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343831713822-31983-0/minicluster-data/master-2/wal/instance
I20260810 06:40:38.615981 31983 external_mini_cluster.cc:1428] Setting key dcd7d4a7688a83840aaf2af03536fca8
I20260810 06:40:38.618908 22635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:38.626745 22635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:40:38 dist-test-slave-4tdj krb5kdc[22293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.252@KRBTEST.COM
Aug 10 06:40:38 dist-test-slave-4tdj krb5kdc[22293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344038, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.252@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:40:38.639555 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.617849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39952 (local address 127.31.59.252:45009)
0810 06:40:38.618685 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:38.618697 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:38.618738 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:40:38.620308 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:38.620329 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:38.620692 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:38.620990 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:38.622754 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.624180 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.626034 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.626741 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.626881 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.631734 (+ 4853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:38.631743 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:38.631753 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:38.631797 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:38.635067 (+ 3270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.635780 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.635787 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.635793 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.635978 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.636401 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.636406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.636408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.636966 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:38.637286 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:38.637769 (+ 483us) server_negotiation.cc:299] Negotiation successful
0810 06:40:38.638355 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":392,"threads_started":1}
I20260810 06:40:38.645910 22639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.629650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:38743 (local address 127.0.0.1:37166)
0810 06:40:38.630451 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.630494 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:38.630720 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:38.631125 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:38.631132 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:38.631828 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:38.632147 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:38.632156 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.633507 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.633511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:38.634625 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:38.634644 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.635593 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.635599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:38.635788 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.636325 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:38.636345 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:38.638681 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:38.642480 (+ 3799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:38.642489 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:38.642496 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:38.643021 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:38.643971 (+ 950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:38.643976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:38.643980 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:38.644180 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:38.644785 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:38.644794 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:38.644972 (+ 178us) client_negotiation.cc:769] Sending connection context
0810 06:40:38.645124 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 06:40:38.645455 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":392,"threads_started":1}
I20260810 06:40:38.646030 22417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.629692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37166 (local address 127.31.59.254:38743)
0810 06:40:38.629928 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:38.629934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:38.630576 (+ 642us) server_negotiation.cc:407] Connection header received
0810 06:40:38.630869 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:38.630874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:38.630942 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:38.631045 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:38.632311 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.633349 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.634785 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.635412 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.635517 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.638895 (+ 3378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:38.638902 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:38.638907 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:38.638952 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:38.642342 (+ 3390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.643159 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.643164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.643168 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.643835 (+ 667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.644331 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.644336 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.644339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.644647 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:38.644816 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:38.645302 (+ 486us) server_negotiation.cc:299] Negotiation successful
0810 06:40:38.645523 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:40:38.647763 22635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:40:38 dist-test-slave-4tdj krb5kdc[22293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
Aug 10 06:40:38 dist-test-slave-4tdj krb5kdc[22293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344038, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.252@KRBTEST.COM for kudu/127.31.59.253@KRBTEST.COM
I20260810 06:40:38.657724 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.643310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37174 (local address 127.31.59.254:38743)
0810 06:40:38.643867 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:38.643872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:38.643894 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:40:38.643948 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:38.643953 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:38.644039 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:38.644176 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:38.645611 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.647013 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.648301 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.648878 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.649020 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.652728 (+ 3708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:38.652732 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:38.652735 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:38.652764 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:38.655318 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.655922 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.655927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.655929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.656044 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.656384 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.656389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.656393 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.656647 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:38.656880 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:38.657186 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 06:40:38.657388 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":240,"threads_started":1}
I20260810 06:40:38.661103 22639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.648556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36263 (local address 127.0.0.1:59846)
0810 06:40:38.648715 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.648734 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:38.648816 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:38.649277 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:38.649281 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:38.649615 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:38.649753 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:38.649762 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.650809 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.650815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:38.651713 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:38.651726 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.652510 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.652516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:38.652709 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.653207 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:38.653224 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:38.655226 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:38.658442 (+ 3216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:38.658448 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:38.658453 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:38.658914 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:38.659397 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:38.659402 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:38.659405 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:38.659603 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:38.660161 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:38.660166 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:38.660321 (+ 155us) client_negotiation.cc:769] Sending connection context
0810 06:40:38.660448 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 06:40:38.660716 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 06:40:38.661183 22530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.648534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59846 (local address 127.31.59.253:36263)
0810 06:40:38.648719 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:38.648725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:38.648783 (+ 58us) server_negotiation.cc:407] Connection header received
0810 06:40:38.648902 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:38.648908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:38.648993 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:38.649093 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:38.649912 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.650666 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.651864 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.652335 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.652456 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.655421 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:38.655426 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:38.655430 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:38.655468 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:38.658311 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.659032 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.659038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.659043 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.659228 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.659706 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.659712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.659716 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.660036 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:38.660179 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:38.660571 (+ 392us) server_negotiation.cc:299] Negotiation successful
0810 06:40:38.660790 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:40:38.662432 22635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:40:38 dist-test-slave-4tdj krb5kdc[22293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.253@KRBTEST.COM
Aug 10 06:40:38 dist-test-slave-4tdj krb5kdc[22293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344038, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.252@KRBTEST.COM for kudu/127.31.59.252@KRBTEST.COM
I20260810 06:40:38.675662 22642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.659446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59850 (local address 127.31.59.253:36263)
0810 06:40:38.659997 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:38.660001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:38.660019 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:40:38.660055 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:38.660058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:38.660106 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:38.660194 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:38.661766 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.662742 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.664325 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.664985 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.665116 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.669849 (+ 4733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:38.669855 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:38.669858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:38.669889 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:38.673040 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.673702 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.673706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.673708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.673840 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.674179 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.674185 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.674188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.674538 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:38.674805 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:38.675046 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 06:40:38.675222 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":264,"threads_started":1}
I20260810 06:40:38.675879 22639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.663184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39956)
0810 06:40:38.663344 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.663361 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:38.663449 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:38.663722 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:38.663726 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:38.664147 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:38.664313 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:38.664324 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.665569 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.665575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:38.666508 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:38.666519 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.667506 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.667510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:38.667619 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.668077 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:38.668093 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:38.670111 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:38.673454 (+ 3343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:38.673459 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:38.673463 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:38.673835 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:38.674225 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:38.674229 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:38.674231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:38.674384 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:38.674986 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:38.674991 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:38.675139 (+ 148us) client_negotiation.cc:769] Sending connection context
0810 06:40:38.675308 (+ 169us) client_negotiation.cc:240] Negotiation successful
0810 06:40:38.675570 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 06:40:38.676201 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.663232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39956 (local address 127.31.59.252:45009)
0810 06:40:38.663395 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:38.663401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:38.663422 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:40:38.663541 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:38.663545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:38.663599 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:38.663677 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:38.664446 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.665436 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.666648 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.667319 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.667511 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.670316 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:38.670324 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:38.670330 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:38.670370 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:38.673336 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.673951 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.673958 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.673962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.674110 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.674499 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.674504 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.674507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.674861 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:38.675005 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:38.675425 (+ 420us) server_negotiation.cc:299] Negotiation successful
0810 06:40:38.675761 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":65}
I20260810 06:40:38.680524 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:40:38.681986 22635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87: Bootstrap starting.
I20260810 06:40:38.686800 22635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:38.688405 22635 log.cc:824] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:38.694392 22635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87: No bootstrap required, opened a new log
I20260810 06:40:38.705479 22635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } }
I20260810 06:40:38.706104 22635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:38.706310 22635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a73e909ba184c8cb8f7951802b60e87, State: Initialized, Role: FOLLOWER
I20260810 06:40:38.707087 22635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } }
I20260810 06:40:38.708509 22657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } } }
I20260810 06:40:38.709280 22657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:40:38.710028 22635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:38.713405 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.686810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37182 (local address 127.31.59.254:38743)
0810 06:40:38.686997 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:38.687003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:38.689754 (+ 2751us) server_negotiation.cc:407] Connection header received
0810 06:40:38.689979 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:38.689986 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:38.690085 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:38.690200 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:38.692980 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.694049 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.696378 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.697060 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.697172 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.704575 (+ 7403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:38.704583 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:38.704588 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:38.704634 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:38.708787 (+ 4153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.710974 (+ 2187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.710980 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.710983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.711144 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.711639 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.711644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.711648 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.712010 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:38.712246 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:38.712833 (+ 587us) server_negotiation.cc:299] Negotiation successful
0810 06:40:38.713067 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:40:38.715204 22642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.686423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59854 (local address 127.31.59.253:36263)
0810 06:40:38.689785 (+ 3362us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:38.689791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:38.699197 (+ 9406us) server_negotiation.cc:407] Connection header received
0810 06:40:38.699704 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:38.699710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:38.699798 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:38.699869 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:38.701973 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.702693 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.704173 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.704557 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.704662 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.707118 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:38.707126 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:38.707130 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:38.707176 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:38.710640 (+ 3464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.711913 (+ 1273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.711920 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.711924 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.712108 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.712666 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.712672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.712677 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.713089 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:38.714359 (+ 1270us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:38.714504 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 06:40:38.714760 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3210}
I20260810 06:40:38.721155 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.686552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39968 (local address 127.31.59.252:45009)
0810 06:40:38.686706 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:38.686712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:38.701979 (+ 15267us) server_negotiation.cc:407] Connection header received
0810 06:40:38.702244 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:38.702250 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:38.702358 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:38.702433 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:38.705437 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.706484 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.710149 (+ 3665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.710767 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.710890 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.714922 (+ 4032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:38.714928 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:38.714932 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:38.714970 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:38.717936 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.718604 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.718609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.718611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.718762 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.719108 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.719112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.719114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.719530 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:38.719766 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:38.720442 (+ 676us) server_negotiation.cc:299] Negotiation successful
0810 06:40:38.720710 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 06:40:38.726588 22673 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:40:38.730580 22673 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:40:38.723701 22535 master_runner.cc:186] Error getting master registration for 127.31.59.254:38743: OK, instance_id { permanent_uuid: "935dae40e80843748c24ef208194bd3a" instance_seqno: 1786344037632924 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:40:38.736104 22535 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:40:38.765652 22412 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } attempt: 4
Aug 10 06:40:38 dist-test-slave-4tdj krb5kdc[22293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344037, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for kudu/127.31.59.252@KRBTEST.COM
I20260810 06:40:38.781755 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.766433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39982)
0810 06:40:38.766769 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.766791 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:38.766900 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:38.767322 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:38.767327 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:38.767876 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:38.768108 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:38.768122 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.769450 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.769456 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:38.770982 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:38.770999 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.771759 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.771766 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:38.771949 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.772565 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:38.772589 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:38.775049 (+ 2460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:38.779163 (+ 4114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:38.779168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:38.779173 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:38.779649 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:38.780042 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:38.780047 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:38.780051 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:38.780219 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:38.780801 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:38.780806 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:38.780970 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 06:40:38.781091 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 06:40:38.781367 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193}
I20260810 06:40:38.781805 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.766744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39982 (local address 127.31.59.252:45009)
0810 06:40:38.766908 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:38.766914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:38.766932 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:40:38.767056 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:38.767062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:38.767168 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:38.767267 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:38.768286 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.769313 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.771131 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.771612 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.771711 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.775224 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:38.775229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:38.775232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:38.775262 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:38.779050 (+ 3788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.779766 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.779771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.779773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.779941 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.780336 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.780341 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.780344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.780690 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:38.780823 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:38.781227 (+ 404us) server_negotiation.cc:299] Negotiation successful
0810 06:40:38.781454 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:40:38.786226 22412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a: Bootstrap starting.
I20260810 06:40:38.789726 22412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:38.791121 22412 log.cc:824] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:38.793732 22412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a: No bootstrap required, opened a new log
I20260810 06:40:38.800638 22412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } }
I20260810 06:40:38.800904 22412 raft_consensus.cc:374] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:38.801007 22412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 935dae40e80843748c24ef208194bd3a, State: Initialized, Role: FOLLOWER
I20260810 06:40:38.801554 22412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } }
I20260810 06:40:38.802438 22679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } } }
I20260810 06:40:38.802830 22679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:40:38.803825 22412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [sys.catalog]: configured and running, proceeding with master startup.
W20260810 06:40:38.813005 22698 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:40:38.813166 22698 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:40:38.812422 22312 master_runner.cc:186] Error getting master registration for 127.31.59.253:36263: OK, instance_id { permanent_uuid: "ee59c7e874514948994d0b6084c93624" instance_seqno: 1786344038106675 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:40:38.824792 22312 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:40:38.838137 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } attempt: 5
Aug 10 06:40:38 dist-test-slave-4tdj krb5kdc[22293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344038, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.253@KRBTEST.COM for kudu/127.31.59.252@KRBTEST.COM
W20260810 06:40:38.847141 22566 tablet.cc:2369] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:40:38.855345 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.838747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:45009 (local address 127.0.0.1:39992)
0810 06:40:38.839015 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:38.839036 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:38.839152 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:38.839499 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:38.839505 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:38.840035 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:38.840259 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:38.840272 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.841682 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.841688 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:38.843159 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:38.843175 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.844141 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.844148 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:38.844378 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.845041 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:38.845066 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:38.847903 (+ 2837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:38.851942 (+ 4039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:38.851948 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:38.851952 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:38.852453 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:38.852868 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:38.852873 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:38.852877 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:38.853040 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:38.854314 (+ 1274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:38.854320 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:38.854478 (+ 158us) client_negotiation.cc:769] Sending connection context
0810 06:40:38.854642 (+ 164us) client_negotiation.cc:240] Negotiation successful
0810 06:40:38.854927 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260810 06:40:38.856456 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:38.838915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39992 (local address 127.31.59.252:45009)
0810 06:40:38.839099 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:38.839104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:38.839124 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:40:38.839270 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:38.839275 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:38.839349 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:38.839457 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:38.840411 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.841536 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.843314 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:38.843983 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:38.844120 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:38.848128 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:38.848139 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:38.848144 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:38.848184 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:38.851801 (+ 3617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.852596 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.852602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.852605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.852750 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:38.853615 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:38.853621 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:38.853624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:38.854063 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:38.854220 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:38.855778 (+ 1558us) server_negotiation.cc:299] Negotiation successful
0810 06:40:38.856044 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:40:38.862528 22523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624: Bootstrap starting.
I20260810 06:40:38.866590 22523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:38.867986 22523 log.cc:824] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:38.872946 22523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624: No bootstrap required, opened a new log
W20260810 06:40:38.873231 22345 tablet.cc:2369] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:40:38.884089 22523 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } }
I20260810 06:40:38.884490 22523 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:38.884605 22523 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee59c7e874514948994d0b6084c93624, State: Initialized, Role: FOLLOWER
I20260810 06:40:38.885275 22523 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } }
I20260810 06:40:38.886248 22703 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } } }
I20260810 06:40:38.886741 22703 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:40:38.887531 22523 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:38.908174 22421 master_runner.cc:263] No leader master found from master ee59c7e874514948994d0b6084c93624
I20260810 06:40:38.908332 22421 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 06:40:38.917027 22720 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:40:38.917268 22720 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 06:40:39.097074 22450 tablet.cc:2369] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:40:39.114835 22657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:40:39.115125 22657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } }
I20260810 06:40:39.116475 22657 leader_election.cc:288] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 935dae40e80843748c24ef208194bd3a (127.31.59.254:38743), ee59c7e874514948994d0b6084c93624 (127.31.59.253:36263)
I20260810 06:40:39.116853 22679 raft_consensus.cc:482] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:40:39.117157 22679 raft_consensus.cc:504] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } }
I20260810 06:40:39.117292 22381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3a73e909ba184c8cb8f7951802b60e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "935dae40e80843748c24ef208194bd3a" is_pre_election: true
I20260810 06:40:39.117857 22381 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a73e909ba184c8cb8f7951802b60e87 in term 0.
I20260810 06:40:39.118549 22562 leader_election.cc:302] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a73e909ba184c8cb8f7951802b60e87, 935dae40e80843748c24ef208194bd3a; no voters:
I20260810 06:40:39.118954 22679 leader_election.cc:288] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee59c7e874514948994d0b6084c93624 (127.31.59.253:36263), 3a73e909ba184c8cb8f7951802b60e87 (127.31.59.252:45009)
I20260810 06:40:39.119655 22657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:40:39.119797 22657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:40:39.119903 22657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:39.119921 22488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "935dae40e80843748c24ef208194bd3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee59c7e874514948994d0b6084c93624" is_pre_election: true
I20260810 06:40:39.120517 22488 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 935dae40e80843748c24ef208194bd3a in term 0.
I20260810 06:40:39.122598 22343 leader_election.cc:302] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 935dae40e80843748c24ef208194bd3a, ee59c7e874514948994d0b6084c93624; no voters:
I20260810 06:40:39.123167 22679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:40:39.123339 22679 raft_consensus.cc:482] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:40:39.123453 22679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:39.123363 22602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "935dae40e80843748c24ef208194bd3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a73e909ba184c8cb8f7951802b60e87" is_pre_election: true
I20260810 06:40:39.126024 22679 raft_consensus.cc:504] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } }
I20260810 06:40:39.126576 22679 leader_election.cc:288] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [CANDIDATE]: Term 1 election: Requested vote from peers ee59c7e874514948994d0b6084c93624 (127.31.59.253:36263), 3a73e909ba184c8cb8f7951802b60e87 (127.31.59.252:45009)
I20260810 06:40:39.126998 22488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "935dae40e80843748c24ef208194bd3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee59c7e874514948994d0b6084c93624"
I20260810 06:40:39.127228 22488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:39.128042 22601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "935dae40e80843748c24ef208194bd3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a73e909ba184c8cb8f7951802b60e87"
I20260810 06:40:39.129042 22487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3a73e909ba184c8cb8f7951802b60e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee59c7e874514948994d0b6084c93624" is_pre_election: true
I20260810 06:40:39.130488 22488 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 935dae40e80843748c24ef208194bd3a in term 1.
I20260810 06:40:39.130964 22343 leader_election.cc:302] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 935dae40e80843748c24ef208194bd3a, ee59c7e874514948994d0b6084c93624; no voters:
I20260810 06:40:39.135874 22657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } }
I20260810 06:40:39.136389 22602 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 935dae40e80843748c24ef208194bd3a in current term 1: Already voted for candidate 3a73e909ba184c8cb8f7951802b60e87 in this term.
I20260810 06:40:39.138917 22381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3a73e909ba184c8cb8f7951802b60e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "935dae40e80843748c24ef208194bd3a"
I20260810 06:40:39.139207 22381 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a73e909ba184c8cb8f7951802b60e87 in current term 1: Already voted for candidate 935dae40e80843748c24ef208194bd3a in this term.
I20260810 06:40:39.139938 22657 leader_election.cc:288] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [CANDIDATE]: Term 1 election: Requested vote from peers 935dae40e80843748c24ef208194bd3a (127.31.59.254:38743), ee59c7e874514948994d0b6084c93624 (127.31.59.253:36263)
I20260810 06:40:39.140164 22679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:40:39.140398 22488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3a73e909ba184c8cb8f7951802b60e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee59c7e874514948994d0b6084c93624"
I20260810 06:40:39.140754 22488 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a73e909ba184c8cb8f7951802b60e87 in current term 1: Already voted for candidate 935dae40e80843748c24ef208194bd3a in this term.
I20260810 06:40:39.141304 22563 leader_election.cc:302] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a73e909ba184c8cb8f7951802b60e87; no voters: 935dae40e80843748c24ef208194bd3a, ee59c7e874514948994d0b6084c93624
I20260810 06:40:39.141803 22657 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 06:40:39.150848 22679 raft_consensus.cc:686] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [term 1 LEADER]: Becoming Leader. State: Replica: 935dae40e80843748c24ef208194bd3a, State: Running, Role: LEADER
I20260810 06:40:39.151443 22679 consensus_queue.cc:237] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } }
I20260810 06:40:39.154206 22724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 935dae40e80843748c24ef208194bd3a. Latest consensus state: current_term: 1 leader_uuid: "935dae40e80843748c24ef208194bd3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } } }
I20260810 06:40:39.154655 22724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:39.156731 22726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:40:39.158468 22726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:40:39.165920 22602 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [term 1 FOLLOWER]: Refusing update from remote peer 935dae40e80843748c24ef208194bd3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:40:39.165918 22488 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624 [term 1 FOLLOWER]: Refusing update from remote peer 935dae40e80843748c24ef208194bd3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:40:39.166886 22679 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:40:39.167228 22724 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:40:39.193063 22657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 935dae40e80843748c24ef208194bd3a. Latest consensus state: current_term: 1 leader_uuid: "935dae40e80843748c24ef208194bd3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } } }
I20260810 06:40:39.193399 22657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:40:39.194185 22703 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 935dae40e80843748c24ef208194bd3a. Latest consensus state: current_term: 1 leader_uuid: "935dae40e80843748c24ef208194bd3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } } }
I20260810 06:40:39.194525 22703 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:40:39.196101 22727 mvcc.cc:205] Tried to move back new op lower bound from 7316865184413532160 to 7316865184368390144. Current Snapshot: MvccSnapshot[applied={T|T < 7316865184413532160}]
I20260810 06:40:39.202720 22724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "935dae40e80843748c24ef208194bd3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } } }
I20260810 06:40:39.203167 22724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:39.204309 22657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "935dae40e80843748c24ef208194bd3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } } }
I20260810 06:40:39.204561 22657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:40:39.204715 22733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "935dae40e80843748c24ef208194bd3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } } }
I20260810 06:40:39.205185 22726 catalog_manager.cc:1347] Generated new cluster ID: b698eaeec4a94452b2cac499ffe937f2
I20260810 06:40:39.205304 22726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:40:39.206863 22730 mvcc.cc:205] Tried to move back new op lower bound from 7316865184413532160 to 7316865184368390144. Current Snapshot: MvccSnapshot[applied={T|T < 7316865184413532160}]
I20260810 06:40:39.205267 22733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:39.214781 22703 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "935dae40e80843748c24ef208194bd3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "935dae40e80843748c24ef208194bd3a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38743 } } peers { permanent_uuid: "ee59c7e874514948994d0b6084c93624" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36263 } } peers { permanent_uuid: "3a73e909ba184c8cb8f7951802b60e87" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 45009 } } }
I20260810 06:40:39.215055 22703 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:40:39.235165 22726 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:40:39.236896 22726 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:40:39.246615 22726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 935dae40e80843748c24ef208194bd3a: Generated new TSK 0
I20260810 06:40:39.248049 22726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:40:39.732697 22673 catalog_manager.cc:1259] Loaded cluster ID: b698eaeec4a94452b2cac499ffe937f2
I20260810 06:40:39.732806 22673 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87: loading cluster ID for follower catalog manager: success
I20260810 06:40:39.735320 22673 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87: acquiring CA information for follower catalog manager: success
I20260810 06:40:39.737120 22673 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3a73e909ba184c8cb8f7951802b60e87: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:40:39.919389 22720 catalog_manager.cc:1259] Loaded cluster ID: b698eaeec4a94452b2cac499ffe937f2
I20260810 06:40:39.919504 22720 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624: loading cluster ID for follower catalog manager: success
I20260810 06:40:39.921965 22720 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624: acquiring CA information for follower catalog manager: success
I20260810 06:40:39.923763 22720 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ee59c7e874514948994d0b6084c93624: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:40:40.324492 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 22312
I20260810 06:40:40.325768 22406 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:40.435351 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 22312
W20260810 06:40:40.463184 22561 connection.cc:535] server connection from 127.0.0.1:39982 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 06:40:40.463538 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 22421
I20260810 06:40:40.464654 22512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:40.567953 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 22421
I20260810 06:40:40.595642 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 22535
I20260810 06:40:40.596798 22629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:40.701850 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 22535
2026-08-10T06:40:40Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4983 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:40:40 dist-test-slave-4tdj krb5kdc[22749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:40:40 dist-test-slave-4tdj krb5kdc[22749](info): set up 2 sockets
Aug 10 06:40:40 dist-test-slave-4tdj krb5kdc[22749](info): commencing operation
krb5kdc: starting...
W20260810 06:40:41.921437 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:40:42 dist-test-slave-4tdj krb5kdc[22749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:40:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:40:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:40:42.164108 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:36063
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:44907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:36063
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:42.502187 22769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:42.502467 22769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:42.502560 22769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:42.509516 22769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:42.509621 22769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:42.509667 22769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:42.509723 22769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:42.509765 22769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:42.519186 22769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:44907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:36063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:36063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:42.520375 22769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:42.521606 22769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:42.532806 22776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:42.532852 22779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:42.532879 22775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:42.533660 22769 server_base.cc:1034] running on GCE node
I20260810 06:40:42.534405 22769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:42.535732 22769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:42.536938 22769 hybrid_clock.cc:648] HybridClock initialized: now 1786344042536897 us; error 30 us; skew 500 ppm
I20260810 06:40:42.537250 22769 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:42 dist-test-slave-4tdj krb5kdc[22749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344042, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:42.540153 22769 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:40:42.541610 22769 webserver.cc:460] Webserver started at http://127.31.59.254:38433/ using document root <none> and password file <none>
I20260810 06:40:42.542240 22769 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:42.542390 22769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:42.542692 22769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:42.545563 22769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "53701e0b524d4a79965785209142a333"
format_stamp: "Formatted at 2026-08-10 06:40:42 on dist-test-slave-4tdj"
server_key: "b2f6aee021922432e6e94f9c40ad83a7"
server_key_iv: "d8af80e8e764c0195baf0d82ec4be26c"
server_key_version: "encryptionkey@0"
I20260810 06:40:42.546330 22769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "53701e0b524d4a79965785209142a333"
format_stamp: "Formatted at 2026-08-10 06:40:42 on dist-test-slave-4tdj"
server_key: "b2f6aee021922432e6e94f9c40ad83a7"
server_key_iv: "d8af80e8e764c0195baf0d82ec4be26c"
server_key_version: "encryptionkey@0"
I20260810 06:40:42.551367 22769 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:40:42.554728 22791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:42.556030 22769 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:40:42.556187 22769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "53701e0b524d4a79965785209142a333"
format_stamp: "Formatted at 2026-08-10 06:40:42 on dist-test-slave-4tdj"
server_key: "b2f6aee021922432e6e94f9c40ad83a7"
server_key_iv: "d8af80e8e764c0195baf0d82ec4be26c"
server_key_version: "encryptionkey@0"
I20260810 06:40:42.556362 22769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:42.582504 22769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:42.586310 22769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:42.586565 22769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:42.603871 22769 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:36063
I20260810 06:40:42.603915 22877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:36063 every 8 connection(s)
I20260810 06:40:42.605120 22769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:40:42.608949 22878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:42.608978 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 22769
I20260810 06:40:42.609197 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:40:42.610134 31983 external_mini_cluster.cc:1428] Setting key 98dc84ca0bb80e18ccc365b66a87a98d
I20260810 06:40:42.623076 22878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333: Bootstrap starting.
Aug 10 06:40:42 dist-test-slave-4tdj krb5kdc[22749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:40:42.626906 22878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:42.628021 22878 log.cc:824] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:42.631407 22878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333: No bootstrap required, opened a new log
I20260810 06:40:42.635318 22884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:42.612006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45366 (local address 127.31.59.254:36063)
0810 06:40:42.612784 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:42.612794 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:42.612833 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:40:42.615244 (+ 2411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:42.615269 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:42.615619 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:42.615932 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:42.617832 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:42.619197 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:42.621143 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:42.621811 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:42.621943 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:42.626971 (+ 5028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:42.626978 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:42.626992 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:42.627043 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:42.630581 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:42.631267 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:42.631276 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:42.631284 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:42.631519 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:42.631933 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:42.631939 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:42.631942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:42.632649 (+ 707us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:42.632980 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:42.633405 (+ 425us) server_negotiation.cc:299] Negotiation successful
0810 06:40:42.633862 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":349,"threads_started":1}
I20260810 06:40:42.639094 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:40:42.641431 22878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53701e0b524d4a79965785209142a333" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36063 } }
I20260810 06:40:42.641918 22878 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:42.642040 22878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53701e0b524d4a79965785209142a333, State: Initialized, Role: FOLLOWER
I20260810 06:40:42.642871 22878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53701e0b524d4a79965785209142a333" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36063 } }
I20260810 06:40:42.643236 22878 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:40:42.643400 22878 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:40:42.643589 22878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:42.645905 22878 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53701e0b524d4a79965785209142a333" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36063 } }
I20260810 06:40:42.646453 22878 leader_election.cc:302] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53701e0b524d4a79965785209142a333; no voters:
I20260810 06:40:42.647034 22878 leader_election.cc:288] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:40:42.647150 22888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:40:42.647807 22888 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333 [term 1 LEADER]: Becoming Leader. State: Replica: 53701e0b524d4a79965785209142a333, State: Running, Role: LEADER
I20260810 06:40:42.648341 22888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53701e0b524d4a79965785209142a333" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36063 } }
I20260810 06:40:42.649240 22878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:42.651248 22890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53701e0b524d4a79965785209142a333" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53701e0b524d4a79965785209142a333" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36063 } } }
I20260810 06:40:42.651275 22891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53701e0b524d4a79965785209142a333. Latest consensus state: current_term: 1 leader_uuid: "53701e0b524d4a79965785209142a333" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53701e0b524d4a79965785209142a333" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36063 } } }
I20260810 06:40:42.651738 22891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333 [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:42.652514 22890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333 [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:42.653240 22896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:40:42.659176 22896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:40:42.667949 22896 catalog_manager.cc:1347] Generated new cluster ID: 5473b4440e194cd3b83989199b23a390
I20260810 06:40:42.668048 22896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:40:42.720002 22896 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:40:42.721335 22896 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:40:42.734674 22896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53701e0b524d4a79965785209142a333: Generated new TSK 0
I20260810 06:40:42.735525 22896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:40:42.752584 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 22769
I20260810 06:40:42.753696 22867 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:42.849105 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 22769
2026-08-10T06:40:42Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2143 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:40:42 dist-test-slave-4tdj krb5kdc[22922](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:40:42 dist-test-slave-4tdj krb5kdc[22922](info): set up 2 sockets
Aug 10 06:40:42 dist-test-slave-4tdj krb5kdc[22922](info): commencing operation
krb5kdc: starting...
W20260810 06:40:44.083875 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:40:44 dist-test-slave-4tdj krb5kdc[22922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:40:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:40:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:40:44.331810 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40613
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:40613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:44.652267 22940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:44.652566 22940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:44.652686 22940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:44.659899 22940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:44.660009 22940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:44.660061 22940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:44.660144 22940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:44.660198 22940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:44.669894 22940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:40613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:44.671161 22940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:44.672412 22940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:44.683527 22949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:44.683607 22951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:44.683568 22953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:44.684607 22940 server_base.cc:1034] running on GCE node
I20260810 06:40:44.685348 22940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:44.686657 22940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:44.687849 22940 hybrid_clock.cc:648] HybridClock initialized: now 1786344044687806 us; error 43 us; skew 500 ppm
I20260810 06:40:44.688169 22940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:44 dist-test-slave-4tdj krb5kdc[22922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344044, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:44.691064 22940 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:40:44.692350 22940 webserver.cc:460] Webserver started at http://127.31.59.254:45513/ using document root <none> and password file <none>
I20260810 06:40:44.692965 22940 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:44.693077 22940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:44.693348 22940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:44.696211 22940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "625f6510ab8e463dbed5383ee32adc0d"
format_stamp: "Formatted at 2026-08-10 06:40:44 on dist-test-slave-4tdj"
server_key: "3d86bbfb462906dfc19ec89d043fdabe"
server_key_iv: "c86dcc0a94220e27d3da9b04daae4de9"
server_key_version: "encryptionkey@0"
I20260810 06:40:44.696892 22940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "625f6510ab8e463dbed5383ee32adc0d"
format_stamp: "Formatted at 2026-08-10 06:40:44 on dist-test-slave-4tdj"
server_key: "3d86bbfb462906dfc19ec89d043fdabe"
server_key_iv: "c86dcc0a94220e27d3da9b04daae4de9"
server_key_version: "encryptionkey@0"
I20260810 06:40:44.701697 22940 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:40:44.704922 22963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:44.706140 22940 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:40:44.706312 22940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "625f6510ab8e463dbed5383ee32adc0d"
format_stamp: "Formatted at 2026-08-10 06:40:44 on dist-test-slave-4tdj"
server_key: "3d86bbfb462906dfc19ec89d043fdabe"
server_key_iv: "c86dcc0a94220e27d3da9b04daae4de9"
server_key_version: "encryptionkey@0"
I20260810 06:40:44.706482 22940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:44.744011 22940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:44.747785 22940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:44.748039 22940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:44.765178 22940 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:40613
I20260810 06:40:44.765233 23042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:40613 every 8 connection(s)
I20260810 06:40:44.766513 22940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:40:44.770354 23044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:44.774988 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 22940
I20260810 06:40:44.775220 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:40:44.776057 31983 external_mini_cluster.cc:1428] Setting key 17ac91d16c032cf5ebb4e2b72e15f094
I20260810 06:40:44.781401 23044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d: Bootstrap starting.
I20260810 06:40:44.785046 23044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:44.786262 23044 log.cc:824] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:44.789512 23044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d: No bootstrap required, opened a new log
Aug 10 06:40:44 dist-test-slave-4tdj krb5kdc[22922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:40:44.797463 23044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625f6510ab8e463dbed5383ee32adc0d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40613 } }
I20260810 06:40:44.797864 23044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:44.797968 23044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 625f6510ab8e463dbed5383ee32adc0d, State: Initialized, Role: FOLLOWER
I20260810 06:40:44.798583 23044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625f6510ab8e463dbed5383ee32adc0d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40613 } }
I20260810 06:40:44.798787 23044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:40:44.798836 23049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:44.778064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53908 (local address 127.31.59.254:40613)
0810 06:40:44.778856 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:44.778867 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:44.778912 (+ 45us) server_negotiation.cc:407] Connection header received
0810 06:40:44.780601 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:44.780622 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:44.780964 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:44.781274 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:44.782993 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:44.784259 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:44.786035 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:44.786857 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:44.787079 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:44.791519 (+ 4440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:44.791527 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:44.791539 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:44.791595 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:44.794814 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:44.795491 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:44.795497 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:44.795503 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:44.795676 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:44.796047 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:44.796052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:44.796057 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:44.796579 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:44.796858 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:44.797270 (+ 412us) server_negotiation.cc:299] Negotiation successful
0810 06:40:44.797657 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":356,"threads_started":1}
I20260810 06:40:44.798928 23044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:40:44.799170 23044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:44.801615 23044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625f6510ab8e463dbed5383ee32adc0d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40613 } }
I20260810 06:40:44.802074 23044 leader_election.cc:302] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 625f6510ab8e463dbed5383ee32adc0d; no voters:
I20260810 06:40:44.802639 23044 leader_election.cc:288] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:40:44.802767 23051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:40:44.803351 23051 raft_consensus.cc:686] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 1 LEADER]: Becoming Leader. State: Replica: 625f6510ab8e463dbed5383ee32adc0d, State: Running, Role: LEADER
I20260810 06:40:44.803823 23051 consensus_queue.cc:237] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625f6510ab8e463dbed5383ee32adc0d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40613 } }
I20260810 06:40:44.804456 23044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:44.806504 23054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 625f6510ab8e463dbed5383ee32adc0d. Latest consensus state: current_term: 1 leader_uuid: "625f6510ab8e463dbed5383ee32adc0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625f6510ab8e463dbed5383ee32adc0d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40613 } } }
I20260810 06:40:44.806846 23054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:44.807969 23053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "625f6510ab8e463dbed5383ee32adc0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625f6510ab8e463dbed5383ee32adc0d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40613 } } }
I20260810 06:40:44.808338 23053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:44.814033 23060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:40:44.818204 23060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:40:44.825307 23060 catalog_manager.cc:1347] Generated new cluster ID: d0f6080ccbc946f7a2421d05a9764806
I20260810 06:40:44.825455 23060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:40:44.846681 23060 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:40:44.848456 23060 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:40:44.863739 23060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d: Generated new TSK 0
I20260810 06:40:44.864696 23060 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
I20260810 06:40:44.933615 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40613
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:45.263641 23082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:45.263900 23082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:45.263991 23082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:45.271093 23082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:45.271194 23082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:45.271303 23082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:40:45.280571 23082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:45.281718 23082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:45.283033 23082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:45.295316 23089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:45.295424 23090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:45.295433 23098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:45.296631 23082 server_base.cc:1034] running on GCE node
I20260810 06:40:45.297186 23082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:45.298048 23082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:45.299247 23082 hybrid_clock.cc:648] HybridClock initialized: now 1786344045299183 us; error 56 us; skew 500 ppm
I20260810 06:40:45.299578 23082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:45 dist-test-slave-4tdj krb5kdc[22922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344045, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:45.302776 23082 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:40:45.304152 23082 webserver.cc:460] Webserver started at http://127.31.59.193:36821/ using document root <none> and password file <none>
I20260810 06:40:45.304750 23082 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:45.304867 23082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:45.305153 23082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:45.307971 23082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "f477755e860340688d4f8322b3af8afb"
format_stamp: "Formatted at 2026-08-10 06:40:45 on dist-test-slave-4tdj"
server_key: "a5a2a6a8cce2a90c5ae4d9240b978591"
server_key_iv: "74399654fcf2085d9c4149b5b839eb06"
server_key_version: "encryptionkey@0"
I20260810 06:40:45.308693 23082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "f477755e860340688d4f8322b3af8afb"
format_stamp: "Formatted at 2026-08-10 06:40:45 on dist-test-slave-4tdj"
server_key: "a5a2a6a8cce2a90c5ae4d9240b978591"
server_key_iv: "74399654fcf2085d9c4149b5b839eb06"
server_key_version: "encryptionkey@0"
I20260810 06:40:45.313853 23082 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260810 06:40:45.317242 23106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:45.318622 23082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 06:40:45.318760 23082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "f477755e860340688d4f8322b3af8afb"
format_stamp: "Formatted at 2026-08-10 06:40:45 on dist-test-slave-4tdj"
server_key: "a5a2a6a8cce2a90c5ae4d9240b978591"
server_key_iv: "74399654fcf2085d9c4149b5b839eb06"
server_key_version: "encryptionkey@0"
I20260810 06:40:45.318935 23082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:45.336886 23082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:45.340706 23082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:45.340993 23082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:45.341931 23082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:40:45.343313 23082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:40:45.343392 23082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:45.343510 23082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:40:45.343570 23082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:45.373664 23082 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:44425
I20260810 06:40:45.373709 23281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:44425 every 8 connection(s)
I20260810 06:40:45.374941 23082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:40:45.380262 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 23082
I20260810 06:40:45.380488 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:40:45.381278 31983 external_mini_cluster.cc:1428] Setting key 8f888c82e6c8832670cef30e21bdafbb
Aug 10 06:40:45 dist-test-slave-4tdj krb5kdc[22922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344045, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:40:45.394627 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:45.377355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:60195 (local address 127.31.59.254:40613)
0810 06:40:45.377814 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:45.377818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:45.378791 (+ 973us) server_negotiation.cc:407] Connection header received
0810 06:40:45.380926 (+ 2135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:45.380930 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:45.380998 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:45.381103 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:45.382930 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:45.383740 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:45.385265 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:45.385656 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:45.385771 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:45.389177 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:45.389182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:45.389185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:45.389217 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:45.391618 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:45.392324 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:45.392328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:45.392330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:45.392449 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:45.392865 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:45.392868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:45.392870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:45.393127 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:45.393272 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:45.393996 (+ 724us) server_negotiation.cc:299] Negotiation successful
0810 06:40:45.394226 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":230,"threads_started":1}
I20260810 06:40:45.395283 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:45.377921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:60195)
0810 06:40:45.378640 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:45.378685 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:45.380625 (+ 1940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:45.381360 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:45.381373 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:45.382041 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:45.382766 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:45.382791 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:45.383896 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:45.383900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:45.385104 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:45.385118 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:45.385885 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:45.385893 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:45.386104 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:45.386750 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:45.386776 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:45.388971 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:45.391761 (+ 2790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:45.391785 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:45.391797 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:45.392196 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:45.392552 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:45.392557 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:45.392561 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:45.392747 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:45.393326 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:45.393331 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:45.393633 (+ 302us) client_negotiation.cc:769] Sending connection context
0810 06:40:45.393852 (+ 219us) client_negotiation.cc:240] Negotiation successful
0810 06:40:45.394208 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":283,"threads_started":1}
I20260810 06:40:45.396711 23282 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40613
I20260810 06:40:45.396982 23282 heartbeater.cc:464] Registering TS with master...
I20260810 06:40:45.397663 23282 heartbeater.cc:511] Master 127.31.59.254:40613 requested a full tablet report, sending...
I20260810 06:40:45.399950 22987 ts_manager.cc:194] Registered new tserver with Master: f477755e860340688d4f8322b3af8afb (127.31.59.193:44425)
I20260810 06:40:45.401841 22987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:60195
WARNING: no policy specified for kudu/127.31.59.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
I20260810 06:40:45.439561 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40613
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:45.757699 23289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:45.757952 23289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:45.758057 23289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:45.765105 23289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:45.765203 23289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:45.765373 23289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:40:45.774595 23289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:45.775702 23289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:45.776927 23289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:45.789245 23303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:45.789285 23307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:45.789310 23305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:45.790171 23289 server_base.cc:1034] running on GCE node
I20260810 06:40:45.790767 23289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:45.791657 23289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:45.792877 23289 hybrid_clock.cc:648] HybridClock initialized: now 1786344045792830 us; error 36 us; skew 500 ppm
I20260810 06:40:45.793187 23289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:45 dist-test-slave-4tdj krb5kdc[22922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344045, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:45.796224 23289 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:40:45.797514 23289 webserver.cc:460] Webserver started at http://127.31.59.194:46329/ using document root <none> and password file <none>
I20260810 06:40:45.798098 23289 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:45.798197 23289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:45.798476 23289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:45.801146 23289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "6b16ba59652c4075a91b93c5f7347cf8"
format_stamp: "Formatted at 2026-08-10 06:40:45 on dist-test-slave-4tdj"
server_key: "4f861bdd3f3f77277202ca850b1e14c1"
server_key_iv: "68fb9931f1a4c7eb74f5692e88dd1e7c"
server_key_version: "encryptionkey@0"
I20260810 06:40:45.801934 23289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "6b16ba59652c4075a91b93c5f7347cf8"
format_stamp: "Formatted at 2026-08-10 06:40:45 on dist-test-slave-4tdj"
server_key: "4f861bdd3f3f77277202ca850b1e14c1"
server_key_iv: "68fb9931f1a4c7eb74f5692e88dd1e7c"
server_key_version: "encryptionkey@0"
I20260810 06:40:45.806967 23289 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260810 06:40:45.810232 23319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:45.811569 23289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:40:45.811717 23289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "6b16ba59652c4075a91b93c5f7347cf8"
format_stamp: "Formatted at 2026-08-10 06:40:45 on dist-test-slave-4tdj"
server_key: "4f861bdd3f3f77277202ca850b1e14c1"
server_key_iv: "68fb9931f1a4c7eb74f5692e88dd1e7c"
server_key_version: "encryptionkey@0"
I20260810 06:40:45.811911 23289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:45.830888 23289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:45.834836 23289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:45.835093 23289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:45.835999 23289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:40:45.837285 23289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:40:45.837364 23289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:45.837495 23289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:40:45.837551 23289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:45.867300 23289 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:40901
I20260810 06:40:45.867341 23500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:40901 every 8 connection(s)
I20260810 06:40:45.868543 23289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:40:45.873329 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 23289
I20260810 06:40:45.873502 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:40:45.873920 31983 external_mini_cluster.cc:1428] Setting key 65ac31f715155d0d5828e0af21343eeb
Aug 10 06:40:45 dist-test-slave-4tdj krb5kdc[22922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344045, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:40:45.888787 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:45.871038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:59117 (local address 127.31.59.254:40613)
0810 06:40:45.871262 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:45.871267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:45.872419 (+ 1152us) server_negotiation.cc:407] Connection header received
0810 06:40:45.874234 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:45.874240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:45.874357 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:45.874445 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:45.876164 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:45.876858 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:45.878973 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:45.879397 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:45.879515 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:45.883285 (+ 3770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:45.883290 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:45.883293 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:45.883324 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:45.885719 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:45.886410 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:45.886416 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:45.886419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:45.886583 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:45.887045 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:45.887050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:45.887054 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:45.887412 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:45.887567 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:45.888228 (+ 661us) server_negotiation.cc:299] Negotiation successful
0810 06:40:45.888454 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 06:40:45.889564 23504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:45.871525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:59117)
0810 06:40:45.872253 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:45.872300 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:45.873945 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:45.874703 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:45.874715 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:45.875283 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:45.875972 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:45.875999 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:45.877000 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:45.877006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:45.878838 (+ 1832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:45.878854 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:45.879617 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:45.879623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:45.879884 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:45.880618 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:45.880652 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:45.883077 (+ 2425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:45.885850 (+ 2773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:45.885859 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:45.885875 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:45.886270 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:45.886726 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:45.886730 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:45.886732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:45.886926 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:45.887555 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:45.887560 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:45.887847 (+ 287us) client_negotiation.cc:769] Sending connection context
0810 06:40:45.888070 (+ 223us) client_negotiation.cc:240] Negotiation successful
0810 06:40:45.888421 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":279,"threads_started":1}
I20260810 06:40:45.891012 23502 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40613
I20260810 06:40:45.891326 23502 heartbeater.cc:464] Registering TS with master...
I20260810 06:40:45.892247 23502 heartbeater.cc:511] Master 127.31.59.254:40613 requested a full tablet report, sending...
I20260810 06:40:45.893811 22987 ts_manager.cc:194] Registered new tserver with Master: 6b16ba59652c4075a91b93c5f7347cf8 (127.31.59.194:40901)
I20260810 06:40:45.894665 22987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:59117
WARNING: no policy specified for kudu/127.31.59.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
I20260810 06:40:45.931339 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40613
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:46.251798 23511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:46.252056 23511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:46.252148 23511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:46.259109 23511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:46.259202 23511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:46.259315 23511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:40:46.268527 23511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:46.269675 23511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:46.270980 23511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:46.283448 23522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:46.283497 23524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:46.283418 23521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:46.284197 23511 server_base.cc:1034] running on GCE node
I20260810 06:40:46.284796 23511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:46.285681 23511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:46.286872 23511 hybrid_clock.cc:648] HybridClock initialized: now 1786344046286815 us; error 45 us; skew 500 ppm
I20260810 06:40:46.287174 23511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:46 dist-test-slave-4tdj krb5kdc[22922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344046, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:46.290143 23511 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:40:46.291576 23511 webserver.cc:460] Webserver started at http://127.31.59.195:34973/ using document root <none> and password file <none>
I20260810 06:40:46.292254 23511 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:46.292359 23511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:46.292598 23511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:46.295404 23511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "1638d375d1d747c296d64d0f08ac66b9"
format_stamp: "Formatted at 2026-08-10 06:40:46 on dist-test-slave-4tdj"
server_key: "2ee0eb3099216307e176a6a3bc397449"
server_key_iv: "a6c1be8a000be2362349c3be24585c41"
server_key_version: "encryptionkey@0"
I20260810 06:40:46.296123 23511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "1638d375d1d747c296d64d0f08ac66b9"
format_stamp: "Formatted at 2026-08-10 06:40:46 on dist-test-slave-4tdj"
server_key: "2ee0eb3099216307e176a6a3bc397449"
server_key_iv: "a6c1be8a000be2362349c3be24585c41"
server_key_version: "encryptionkey@0"
I20260810 06:40:46.300829 23511 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:40:46.304193 23536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:46.305438 23511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:40:46.305575 23511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "1638d375d1d747c296d64d0f08ac66b9"
format_stamp: "Formatted at 2026-08-10 06:40:46 on dist-test-slave-4tdj"
server_key: "2ee0eb3099216307e176a6a3bc397449"
server_key_iv: "a6c1be8a000be2362349c3be24585c41"
server_key_version: "encryptionkey@0"
I20260810 06:40:46.305761 23511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:46.321842 23511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:46.325547 23511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:46.325811 23511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:46.326716 23511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:40:46.327992 23511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:40:46.328070 23511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:46.328194 23511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:40:46.328249 23511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:46.358150 23511 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:40593
I20260810 06:40:46.358189 23714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:40593 every 8 connection(s)
I20260810 06:40:46.359377 23511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:40:46.363220 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 23511
I20260810 06:40:46.363418 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:40:46.363992 31983 external_mini_cluster.cc:1428] Setting key 04cac11ab30b492dcb5c8c8996135e63
Aug 10 06:40:46 dist-test-slave-4tdj krb5kdc[22922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344046, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:40:46.378849 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:46.361818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:44813 (local address 127.31.59.254:40613)
0810 06:40:46.362037 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:46.362044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:46.363223 (+ 1179us) server_negotiation.cc:407] Connection header received
0810 06:40:46.365447 (+ 2224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:46.365450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:46.365530 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:46.365610 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:46.367318 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.368070 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.369490 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.369913 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.370003 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:46.373201 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:46.373207 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:46.373211 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:46.373254 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:46.375756 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:46.376541 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:46.376545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:46.376547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:46.376677 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:46.377124 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:46.377127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:46.377130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:46.377390 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:46.377520 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:46.378363 (+ 843us) server_negotiation.cc:299] Negotiation successful
0810 06:40:46.378568 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 06:40:46.379701 23718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:46.362252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:44813)
0810 06:40:46.363049 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:46.363098 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:46.365150 (+ 2052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:46.365859 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:46.365870 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:46.366479 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:46.367166 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:46.367188 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.368209 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.368213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:46.369387 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:46.369398 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.370116 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.370121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:46.370340 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:46.370898 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:46.370918 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:46.372993 (+ 2075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:46.375885 (+ 2892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:46.375894 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:46.375908 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:46.376444 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:46.376767 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:46.376771 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:46.376774 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:46.377008 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:46.377522 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:46.377530 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:46.377890 (+ 360us) client_negotiation.cc:769] Sending connection context
0810 06:40:46.378206 (+ 316us) client_negotiation.cc:240] Negotiation successful
0810 06:40:46.378643 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":267,"threads_started":1}
I20260810 06:40:46.380867 23715 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40613
I20260810 06:40:46.381119 23715 heartbeater.cc:464] Registering TS with master...
I20260810 06:40:46.381791 23715 heartbeater.cc:511] Master 127.31.59.254:40613 requested a full tablet report, sending...
I20260810 06:40:46.383116 22987 ts_manager.cc:194] Registered new tserver with Master: 1638d375d1d747c296d64d0f08ac66b9 (127.31.59.195:40593)
I20260810 06:40:46.383884 22987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:44813
I20260810 06:40:46.391119 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:40:46.404773 23282 heartbeater.cc:503] Master 127.31.59.254:40613 was elected leader, sending a full tablet report...
I20260810 06:40:46.407254 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:46.395481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53922 (local address 127.31.59.254:40613)
0810 06:40:46.395651 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:46.395655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:46.396061 (+ 406us) server_negotiation.cc:407] Connection header received
0810 06:40:46.396198 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:46.396201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:46.396257 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:46.396329 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:46.397648 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.398370 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.399638 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.400065 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.400152 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:46.402183 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:46.402188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:46.402191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:46.402219 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:46.404702 (+ 2483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:46.405400 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:46.405405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:46.405409 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:46.405561 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:46.405955 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:46.405958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:46.405960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:46.406359 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:46.406618 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:46.406838 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 06:40:46.406987 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 06:40:46.414933 22987 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:40:46.421509 22987 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:40:46.466516 23743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:46.432886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43074 (local address 127.31.59.194:40901)
0810 06:40:46.457057 (+ 24171us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:46.457068 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:46.458055 (+ 987us) server_negotiation.cc:407] Connection header received
0810 06:40:46.459266 (+ 1211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:46.459275 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:46.459525 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:46.459738 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:40:46.460805 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.462147 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.464348 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.465513 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.465651 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:46.465851 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:46.466023 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 06:40:46.466190 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23961,"thread_start_us":24181,"threads_started":1}
I20260810 06:40:46.466563 23740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:46.432461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:40901 (local address 127.0.0.1:43074)
0810 06:40:46.457925 (+ 25464us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:46.457945 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:46.459042 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:46.460034 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:46.460038 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:46.460081 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:40:46.460626 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:46.460639 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.462327 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.462332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:46.464183 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:46.464196 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.465689 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.465693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:46.465807 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:46.465839 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:40:46.465946 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 06:40:46.466123 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25326,"mutex_wait_us":111,"spinlock_wait_cycles":7168,"thread_start_us":26323,"threads_started":1}
I20260810 06:40:46.467646 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:46.432819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:44425 (local address 127.0.0.1:55676)
0810 06:40:46.459072 (+ 26253us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:46.459087 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:46.459183 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:46.459801 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:46.459808 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:46.459838 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:40:46.460329 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:46.460343 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.462234 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.462239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:46.464800 (+ 2561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:46.464815 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.466866 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.466873 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:46.467036 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:46.467071 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 06:40:46.467163 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 06:40:46.467283 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26125,"mutex_wait_us":52,"thread_start_us":25208,"threads_started":1}
I20260810 06:40:46.467694 23744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:46.433398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55676 (local address 127.31.59.193:44425)
0810 06:40:46.457324 (+ 23926us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:46.457334 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:46.459302 (+ 1968us) server_negotiation.cc:407] Connection header received
0810 06:40:46.459393 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:46.459400 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:46.459577 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:46.459750 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:40:46.460525 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.462102 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.464963 (+ 2861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.466371 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.466505 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:46.466721 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:46.467245 (+ 524us) server_negotiation.cc:299] Negotiation successful
0810 06:40:46.467399 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23731,"thread_start_us":23934,"threads_started":1}
I20260810 06:40:46.468547 23731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:46.432314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:40593 (local address 127.0.0.1:58396)
0810 06:40:46.457862 (+ 25548us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:46.457919 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:46.458790 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:46.460011 (+ 1221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:46.460016 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:46.460052 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:40:46.460572 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:46.460588 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.462498 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.462503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:46.464801 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:46.464815 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.467793 (+ 2978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.467800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:46.467970 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:46.468002 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:40:46.468089 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 06:40:46.468196 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25243,"mutex_wait_us":50,"spinlock_wait_cycles":1280,"thread_start_us":25770,"threads_started":1}
I20260810 06:40:46.469233 23737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:46.432560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58396 (local address 127.31.59.195:40593)
0810 06:40:46.459435 (+ 26875us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:46.459440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:46.459464 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:40:46.459534 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:46.459539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:46.459710 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:46.459890 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:40:46.460789 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.462375 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.465952 (+ 3577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.467329 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.467446 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:46.467651 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:46.468771 (+ 1120us) server_negotiation.cc:299] Negotiation successful
0810 06:40:46.468923 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26730,"thread_start_us":26583,"threads_started":1}
I20260810 06:40:46.470665 23398 tablet_service.cc:1467] Processing CreateTablet for tablet 59df50b4a2144f9585931b08576019d1 (DEFAULT_TABLE table=test-table [id=f26b7511627142a381a33e62e09a38fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:40:46.472173 23398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59df50b4a2144f9585931b08576019d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:46.471731 23177 tablet_service.cc:1467] Processing CreateTablet for tablet 59df50b4a2144f9585931b08576019d1 (DEFAULT_TABLE table=test-table [id=f26b7511627142a381a33e62e09a38fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:40:46.472122 23620 tablet_service.cc:1467] Processing CreateTablet for tablet 59df50b4a2144f9585931b08576019d1 (DEFAULT_TABLE table=test-table [id=f26b7511627142a381a33e62e09a38fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:40:46.473111 23177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59df50b4a2144f9585931b08576019d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:46.473290 23620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59df50b4a2144f9585931b08576019d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:46.484099 23745 tablet_bootstrap.cc:492] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9: Bootstrap starting.
I20260810 06:40:46.486438 23747 tablet_bootstrap.cc:492] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8: Bootstrap starting.
I20260810 06:40:46.487625 23745 tablet_bootstrap.cc:654] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:46.488914 23745 log.cc:824] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:46.489951 23747 tablet_bootstrap.cc:654] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:46.491295 23747 log.cc:824] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:46.492076 23748 tablet_bootstrap.cc:492] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb: Bootstrap starting.
I20260810 06:40:46.492163 23745 tablet_bootstrap.cc:492] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9: No bootstrap required, opened a new log
I20260810 06:40:46.492442 23745 ts_tablet_manager.cc:1397] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 06:40:46.494109 23747 tablet_bootstrap.cc:492] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8: No bootstrap required, opened a new log
I20260810 06:40:46.494385 23747 ts_tablet_manager.cc:1397] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.003s
I20260810 06:40:46.495458 23748 tablet_bootstrap.cc:654] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:46.496591 23748 log.cc:824] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:46.499553 23748 tablet_bootstrap.cc:492] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb: No bootstrap required, opened a new log
I20260810 06:40:46.499806 23748 ts_tablet_manager.cc:1397] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260810 06:40:46.501389 23747 raft_consensus.cc:348] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } } peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } }
I20260810 06:40:46.501703 23747 raft_consensus.cc:374] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:46.501796 23747 raft_consensus.cc:729] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b16ba59652c4075a91b93c5f7347cf8, State: Initialized, Role: FOLLOWER
I20260810 06:40:46.502365 23747 consensus_queue.cc:260] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } } peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } }
I20260810 06:40:46.502878 23745 raft_consensus.cc:348] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } } peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } }
I20260810 06:40:46.503342 23745 raft_consensus.cc:374] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:46.503484 23745 raft_consensus.cc:729] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1638d375d1d747c296d64d0f08ac66b9, State: Initialized, Role: FOLLOWER
I20260810 06:40:46.503762 23502 heartbeater.cc:503] Master 127.31.59.254:40613 was elected leader, sending a full tablet report...
I20260810 06:40:46.504230 23747 ts_tablet_manager.cc:1428] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8: Time spent starting tablet: real 0.010s user 0.005s sys 0.004s
I20260810 06:40:46.504084 23745 consensus_queue.cc:260] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } } peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } }
I20260810 06:40:46.505092 23715 heartbeater.cc:503] Master 127.31.59.254:40613 was elected leader, sending a full tablet report...
I20260810 06:40:46.507568 23745 ts_tablet_manager.cc:1428] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260810 06:40:46.510602 23755 raft_consensus.cc:482] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:40:46.510895 23755 raft_consensus.cc:504] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } } peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } }
I20260810 06:40:46.511601 23748 raft_consensus.cc:348] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } } peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } }
I20260810 06:40:46.511916 23748 raft_consensus.cc:374] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:46.512001 23748 raft_consensus.cc:729] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f477755e860340688d4f8322b3af8afb, State: Initialized, Role: FOLLOWER
I20260810 06:40:46.512542 23755 leader_election.cc:288] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b16ba59652c4075a91b93c5f7347cf8 (127.31.59.194:40901), f477755e860340688d4f8322b3af8afb (127.31.59.193:44425)
I20260810 06:40:46.512562 23748 consensus_queue.cc:260] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } } peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } }
I20260810 06:40:46.516043 23748 ts_tablet_manager.cc:1428] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260810 06:40:46.520007 23743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:46.513466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:34349 (local address 127.31.59.194:40901)
0810 06:40:46.513673 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:46.513679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:46.513698 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:40:46.513761 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:46.513766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:46.513837 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:46.513919 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:40:46.514660 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.515729 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.518101 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.518980 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.519149 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:46.519222 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:46.519635 (+ 413us) server_negotiation.cc:299] Negotiation successful
0810 06:40:46.519762 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:40:46.520044 23718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:46.512742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:40901 (local address 127.31.59.195:34349)
0810 06:40:46.513126 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:46.513148 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:46.513264 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:46.514090 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:46.514096 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:46.514174 (+ 78us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:40:46.514495 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:46.514507 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.515893 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.515898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:46.517908 (+ 2010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:46.517922 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.519284 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.519290 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:46.519475 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:46.519509 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 06:40:46.519611 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 06:40:46.519746 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":7,"spinlock_wait_cycles":21248}
I20260810 06:40:46.520460 23744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:46.513160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:60381 (local address 127.31.59.193:44425)
0810 06:40:46.513350 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:46.513357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:46.513797 (+ 440us) server_negotiation.cc:407] Connection header received
0810 06:40:46.513937 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:46.513942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:46.514001 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:46.514077 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:40:46.514764 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.515889 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.518348 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.519587 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.519737 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:46.519799 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:46.520142 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 06:40:46.520267 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:40:46.520507 23756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:46.512815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:44425 (local address 127.31.59.195:60381)
0810 06:40:46.513704 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:46.513719 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:46.513834 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:46.514097 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:46.514101 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:46.514125 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:40:46.514576 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:46.514586 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.516169 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.516174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:46.518151 (+ 1977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:46.518163 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.519772 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.519778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:46.519954 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:46.519985 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:40:46.520089 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 06:40:46.520216 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"mutex_wait_us":40,"thread_start_us":591,"threads_started":1}
I20260810 06:40:46.521421 23214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59df50b4a2144f9585931b08576019d1" candidate_uuid: "1638d375d1d747c296d64d0f08ac66b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f477755e860340688d4f8322b3af8afb" is_pre_election: true
I20260810 06:40:46.521421 23425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59df50b4a2144f9585931b08576019d1" candidate_uuid: "1638d375d1d747c296d64d0f08ac66b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b16ba59652c4075a91b93c5f7347cf8" is_pre_election: true
I20260810 06:40:46.521934 23425 raft_consensus.cc:2454] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1638d375d1d747c296d64d0f08ac66b9 in term 0.
I20260810 06:40:46.521934 23214 raft_consensus.cc:2454] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1638d375d1d747c296d64d0f08ac66b9 in term 0.
I20260810 06:40:46.522598 23538 leader_election.cc:302] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1638d375d1d747c296d64d0f08ac66b9, 6b16ba59652c4075a91b93c5f7347cf8; no voters:
I20260810 06:40:46.522928 23755 raft_consensus.cc:2781] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:40:46.523074 23755 raft_consensus.cc:482] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:40:46.523188 23755 raft_consensus.cc:3037] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:46.525328 23755 raft_consensus.cc:504] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } } peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } }
I20260810 06:40:46.525861 23755 leader_election.cc:288] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [CANDIDATE]: Term 1 election: Requested vote from peers 6b16ba59652c4075a91b93c5f7347cf8 (127.31.59.194:40901), f477755e860340688d4f8322b3af8afb (127.31.59.193:44425)
I20260810 06:40:46.526336 23214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59df50b4a2144f9585931b08576019d1" candidate_uuid: "1638d375d1d747c296d64d0f08ac66b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f477755e860340688d4f8322b3af8afb"
I20260810 06:40:46.526284 23425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59df50b4a2144f9585931b08576019d1" candidate_uuid: "1638d375d1d747c296d64d0f08ac66b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b16ba59652c4075a91b93c5f7347cf8"
I20260810 06:40:46.526499 23214 raft_consensus.cc:3037] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:46.526528 23425 raft_consensus.cc:3037] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:46.528636 23214 raft_consensus.cc:2454] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1638d375d1d747c296d64d0f08ac66b9 in term 1.
I20260810 06:40:46.529073 23541 leader_election.cc:302] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1638d375d1d747c296d64d0f08ac66b9, f477755e860340688d4f8322b3af8afb; no voters:
I20260810 06:40:46.529139 23425 raft_consensus.cc:2454] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1638d375d1d747c296d64d0f08ac66b9 in term 1.
I20260810 06:40:46.529374 23755 raft_consensus.cc:2781] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:40:46.529814 23755 raft_consensus.cc:686] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [term 1 LEADER]: Becoming Leader. State: Replica: 1638d375d1d747c296d64d0f08ac66b9, State: Running, Role: LEADER
I20260810 06:40:46.530247 23755 consensus_queue.cc:237] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } } peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } }
I20260810 06:40:46.534857 22984 catalog_manager.cc:5495] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1638d375d1d747c296d64d0f08ac66b9 (127.31.59.195). New cstate: current_term: 1 leader_uuid: "1638d375d1d747c296d64d0f08ac66b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } health_report { overall_health: UNKNOWN } } }
I20260810 06:40:46.565099 23737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:46.559848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58406 (local address 127.31.59.195:40593)
0810 06:40:46.560071 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:46.560076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:46.560200 (+ 124us) server_negotiation.cc:407] Connection header received
0810 06:40:46.560373 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:46.560376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:46.560452 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:46.560600 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:40:46.560963 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.561689 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.563137 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.563523 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.563610 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:46.564170 (+ 560us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:40:46.564648 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:40:46.564760 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:46.564847 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 06:40:46.564907 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 06:40:46.572764 23425 raft_consensus.cc:1261] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8 [term 1 FOLLOWER]: Refusing update from remote peer 1638d375d1d747c296d64d0f08ac66b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:40:46.572759 23214 raft_consensus.cc:1261] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb [term 1 FOLLOWER]: Refusing update from remote peer 1638d375d1d747c296d64d0f08ac66b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:40:46.573680 23759 consensus_queue.cc:1035] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:40:46.573989 23755 consensus_queue.cc:1035] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:40:46.586211 23770 mvcc.cc:205] Tried to move back new op lower bound from 7316865214752808960 to 7316865214591692800. Current Snapshot: MvccSnapshot[applied={T|T < 7316865214752808960}]
I20260810 06:40:46.587370 23763 mvcc.cc:205] Tried to move back new op lower bound from 7316865214752808960 to 7316865214591692800. Current Snapshot: MvccSnapshot[applied={T|T < 7316865214752808960}]
I20260810 06:40:46.601241 22984 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:40:46.601559 22984 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:40:46.605537 22984 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d: Sending DeleteTablet for 3 replicas of tablet 59df50b4a2144f9585931b08576019d1
I20260810 06:40:46.606789 23620 tablet_service.cc:1514] Processing DeleteTablet for tablet 59df50b4a2144f9585931b08576019d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:40:46 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:58396
I20260810 06:40:46.606941 23177 tablet_service.cc:1514] Processing DeleteTablet for tablet 59df50b4a2144f9585931b08576019d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:40:46 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:55676
I20260810 06:40:46.606993 23398 tablet_service.cc:1514] Processing DeleteTablet for tablet 59df50b4a2144f9585931b08576019d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:40:46 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:43074
I20260810 06:40:46.609565 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 23082
I20260810 06:40:46.619549 23781 tablet_replica.cc:330] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9: stopping tablet replica
I20260810 06:40:46.620151 23781 raft_consensus.cc:2229] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:40:46.620400 23783 tablet_replica.cc:330] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8: stopping tablet replica
I20260810 06:40:46.621040 23781 raft_consensus.cc:2258] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:40:46.621796 23783 raft_consensus.cc:2229] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:40:46.622416 23783 raft_consensus.cc:2258] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:40:46.624131 23782 tablet_replica.cc:330] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb: stopping tablet replica
I20260810 06:40:46.624648 23782 raft_consensus.cc:2229] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:40:46.624785 23781 ts_tablet_manager.cc:1905] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:40:46.624831 23783 ts_tablet_manager.cc:1905] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:40:46.625226 23782 raft_consensus.cc:2258] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:40:46.627322 23782 ts_tablet_manager.cc:1905] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:40:46.631759 23783 ts_tablet_manager.cc:1918] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:40:46.631978 23783 log.cc:1196] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/59df50b4a2144f9585931b08576019d1
I20260810 06:40:46.632594 23783 ts_tablet_manager.cc:1939] T 59df50b4a2144f9585931b08576019d1 P 6b16ba59652c4075a91b93c5f7347cf8: Deleting consensus metadata
I20260810 06:40:46.634159 23782 ts_tablet_manager.cc:1918] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:40:46.634421 23782 log.cc:1196] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/59df50b4a2144f9585931b08576019d1
I20260810 06:40:46.634507 22966 catalog_manager.cc:4841] TS 6b16ba59652c4075a91b93c5f7347cf8 (127.31.59.194:40901): tablet 59df50b4a2144f9585931b08576019d1 (table test-table [id=f26b7511627142a381a33e62e09a38fe]) successfully deleted
I20260810 06:40:46.635070 23782 ts_tablet_manager.cc:1939] T 59df50b4a2144f9585931b08576019d1 P f477755e860340688d4f8322b3af8afb: Deleting consensus metadata
I20260810 06:40:46.636432 22968 catalog_manager.cc:4841] TS f477755e860340688d4f8322b3af8afb (127.31.59.193:44425): tablet 59df50b4a2144f9585931b08576019d1 (table test-table [id=f26b7511627142a381a33e62e09a38fe]) successfully deleted
I20260810 06:40:46.641031 23781 ts_tablet_manager.cc:1918] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 10 06:40:46 dist-test-slave-4tdj krb5kdc[22922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:40:46.641238 23781 log.cc:1196] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/59df50b4a2144f9585931b08576019d1
I20260810 06:40:46.641866 23781 ts_tablet_manager.cc:1939] T 59df50b4a2144f9585931b08576019d1 P 1638d375d1d747c296d64d0f08ac66b9: Deleting consensus metadata
I20260810 06:40:46.643653 22971 catalog_manager.cc:4841] TS 1638d375d1d747c296d64d0f08ac66b9 (127.31.59.195:40593): tablet 59df50b4a2144f9585931b08576019d1 (table test-table [id=f26b7511627142a381a33e62e09a38fe]) successfully deleted
I20260810 06:40:46.648056 23744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:46.610998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55680 (local address 127.31.59.193:44425)
0810 06:40:46.611180 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:46.611186 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:46.631502 (+ 20316us) server_negotiation.cc:407] Connection header received
0810 06:40:46.632079 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:46.632086 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:46.632183 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:46.632296 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:46.634461 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.635632 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.637605 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.638011 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.638125 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:46.642216 (+ 4091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:46.642229 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:46.642241 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:46.642273 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:46.645050 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:46.645999 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:46.646006 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:46.646008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:46.646168 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:46.646620 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:46.646623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:46.646626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:46.646974 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:46.647237 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:46.647595 (+ 358us) server_negotiation.cc:299] Negotiation successful
0810 06:40:46.647770 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:40:46.648727 23272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:46.772850 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 23082
I20260810 06:40:46.793109 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 23289
Aug 10 06:40:46 dist-test-slave-4tdj krb5kdc[22922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.194@KRBTEST.COM
I20260810 06:40:46.810391 23743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:46.793922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43084 (local address 127.31.59.194:40901)
0810 06:40:46.794090 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:46.794094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:46.794113 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:40:46.794465 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:46.794471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:46.794574 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:46.794666 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:46.796307 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.797072 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.798899 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.799285 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.799377 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:46.804421 (+ 5044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:46.804436 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:46.804453 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:46.804489 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:46.807443 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:46.808065 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:46.808072 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:46.808075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:46.808201 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:46.808592 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:46.808598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:46.808602 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:46.809132 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:46.809452 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:46.809764 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 06:40:46.809997 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:40:46.811384 23495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:46.897173 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 23289
I20260810 06:40:46.923130 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 23511
Aug 10 06:40:46 dist-test-slave-4tdj krb5kdc[22922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.195@KRBTEST.COM
I20260810 06:40:46.939109 23737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:46.923939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58408 (local address 127.31.59.195:40593)
0810 06:40:46.924127 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:46.924133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:46.924155 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:40:46.924340 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:46.924344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:46.924410 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:46.924507 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:46.925786 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.926692 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.927911 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:46.928370 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:46.928477 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:46.933095 (+ 4618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:46.933107 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:46.933122 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:46.933155 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:46.936005 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:46.936886 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:46.936894 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:46.936896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:46.937084 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:46.937532 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:46.937536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:46.937539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:46.937907 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:46.938193 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:46.938596 (+ 403us) server_negotiation.cc:299] Negotiation successful
0810 06:40:46.938767 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:40:46.939833 23708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:47.065238 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 23511
I20260810 06:40:47.089910 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 22940
I20260810 06:40:47.091035 23036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:47.199512 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 22940
I20260810 06:40:47.224658 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40613
--webserver_interface=127.31.59.254
--webserver_port=45513
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:40613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:47.544785 23793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:47.545087 23793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:47.545207 23793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:47.552313 23793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:47.552419 23793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:47.552500 23793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:47.552584 23793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:47.552640 23793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:47.562029 23793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:40613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=45513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:47.563284 23793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:47.564615 23793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:47.587595 23802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:47.587595 23803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:47.587623 23805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:47.588507 23793 server_base.cc:1034] running on GCE node
I20260810 06:40:47.589324 23793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:47.590854 23793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:47.592056 23793 hybrid_clock.cc:648] HybridClock initialized: now 1786344047591997 us; error 48 us; skew 500 ppm
I20260810 06:40:47.592432 23793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:47 dist-test-slave-4tdj krb5kdc[22922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344047, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:47.598444 23793 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:40:47.600070 23793 webserver.cc:460] Webserver started at http://127.31.59.254:45513/ using document root <none> and password file <none>
I20260810 06:40:47.600796 23793 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:47.600929 23793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:47.606747 23793 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 06:40:47.609586 23816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:47.610965 23793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:40:47.611122 23793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "625f6510ab8e463dbed5383ee32adc0d"
format_stamp: "Formatted at 2026-08-10 06:40:44 on dist-test-slave-4tdj"
server_key: "3d86bbfb462906dfc19ec89d043fdabe"
server_key_iv: "c86dcc0a94220e27d3da9b04daae4de9"
server_key_version: "encryptionkey@0"
I20260810 06:40:47.612140 23793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:47.641099 23793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:47.645459 23793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:47.645727 23793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:47.664029 23793 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:40613
I20260810 06:40:47.664050 23891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:40613 every 8 connection(s)
I20260810 06:40:47.665524 23793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:40:47.666906 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 23793
I20260810 06:40:47.667930 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:44425
--local_ip_for_outbound_sockets=127.31.59.193
--tserver_master_addrs=127.31.59.254:40613
--webserver_port=36821
--webserver_interface=127.31.59.193
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:40:47.674816 23892 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:40:47.679219 23892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d: Bootstrap starting.
I20260810 06:40:47.699208 23892 log.cc:824] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:47.711423 23892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:40:47.711921 23892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d: Bootstrap complete.
I20260810 06:40:47.721182 23892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625f6510ab8e463dbed5383ee32adc0d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40613 } }
I20260810 06:40:47.722041 23892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 625f6510ab8e463dbed5383ee32adc0d, State: Initialized, Role: FOLLOWER
I20260810 06:40:47.722693 23892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625f6510ab8e463dbed5383ee32adc0d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40613 } }
I20260810 06:40:47.722920 23892 raft_consensus.cc:388] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:40:47.723048 23892 raft_consensus.cc:482] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:40:47.723222 23892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:40:47.726961 23892 raft_consensus.cc:504] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625f6510ab8e463dbed5383ee32adc0d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40613 } }
I20260810 06:40:47.727487 23892 leader_election.cc:302] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 625f6510ab8e463dbed5383ee32adc0d; no voters:
I20260810 06:40:47.728084 23892 leader_election.cc:288] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:40:47.728228 23896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:40:47.729410 23896 raft_consensus.cc:686] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 2 LEADER]: Becoming Leader. State: Replica: 625f6510ab8e463dbed5383ee32adc0d, State: Running, Role: LEADER
I20260810 06:40:47.730038 23896 consensus_queue.cc:237] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625f6510ab8e463dbed5383ee32adc0d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40613 } }
I20260810 06:40:47.730362 23892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:47.732489 23898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 625f6510ab8e463dbed5383ee32adc0d. Latest consensus state: current_term: 2 leader_uuid: "625f6510ab8e463dbed5383ee32adc0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625f6510ab8e463dbed5383ee32adc0d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40613 } } }
I20260810 06:40:47.732906 23898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:47.733435 23897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "625f6510ab8e463dbed5383ee32adc0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625f6510ab8e463dbed5383ee32adc0d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40613 } } }
I20260810 06:40:47.733706 23897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:47.738328 23906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:40:47.749176 23906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:40:47.751144 23906 catalog_manager.cc:1259] Loaded cluster ID: d0f6080ccbc946f7a2421d05a9764806
I20260810 06:40:47.751236 23906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:40:47.753494 23906 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 06:40:47.764051 23906 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 06:40:47.770670 23906 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:40:47.771888 23906 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d: Loaded TSK: 0
I20260810 06:40:47.772925 23906 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:40:48.036054 23894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:48.036321 23894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:48.036453 23894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:48.043653 23894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:48.043768 23894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:48.043913 23894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:40:48.053303 23894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:44425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=36821
--tserver_master_addrs=127.31.59.254:40613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:48.054629 23894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:48.055930 23894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:48.069762 23933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:48.069726 23935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:48.070473 23939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:48.070706 23894 server_base.cc:1034] running on GCE node
I20260810 06:40:48.071287 23894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:48.072245 23894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:48.073458 23894 hybrid_clock.cc:648] HybridClock initialized: now 1786344048073414 us; error 37 us; skew 500 ppm
I20260810 06:40:48.073755 23894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:48 dist-test-slave-4tdj krb5kdc[22922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344048, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:48.076747 23894 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:40:48.078171 23894 webserver.cc:460] Webserver started at http://127.31.59.193:36821/ using document root <none> and password file <none>
I20260810 06:40:48.078830 23894 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:48.078954 23894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:48.084039 23894 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 06:40:48.086673 23954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:48.087934 23894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:40:48.088068 23894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "f477755e860340688d4f8322b3af8afb"
format_stamp: "Formatted at 2026-08-10 06:40:45 on dist-test-slave-4tdj"
server_key: "a5a2a6a8cce2a90c5ae4d9240b978591"
server_key_iv: "74399654fcf2085d9c4149b5b839eb06"
server_key_version: "encryptionkey@0"
I20260810 06:40:48.088963 23894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:48.123055 23894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:48.127390 23894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:48.127656 23894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:48.128700 23894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:40:48.130085 23894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:40:48.130189 23894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:48.130309 23894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:40:48.130384 23894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260810 06:40:48.160668 23894 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:44425
I20260810 06:40:48.160717 24131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:44425 every 8 connection(s)
I20260810 06:40:48.162099 23894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:40:48.166395 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 23894
I20260810 06:40:48.167171 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:40901
--local_ip_for_outbound_sockets=127.31.59.194
--tserver_master_addrs=127.31.59.254:40613
--webserver_port=46329
--webserver_interface=127.31.59.194
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:40:48 dist-test-slave-4tdj krb5kdc[22922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344048, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:40:48.186645 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:48.165096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:45303)
0810 06:40:48.165955 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:48.166013 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:48.168255 (+ 2242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:48.169563 (+ 1308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:48.169573 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:48.170241 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:48.170984 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:48.171007 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:48.172580 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:48.172586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:48.174351 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:48.174365 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:48.175329 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:48.175336 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:48.175585 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:48.176250 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:48.176282 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:48.178811 (+ 2529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:48.182749 (+ 3938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:48.182758 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:48.182773 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:48.183231 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:48.183689 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:48.183692 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:48.183695 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:48.183864 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:48.184727 (+ 863us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:48.184733 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:48.185009 (+ 276us) client_negotiation.cc:769] Sending connection context
0810 06:40:48.185242 (+ 233us) client_negotiation.cc:240] Negotiation successful
0810 06:40:48.185617 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":302,"threads_started":1}
I20260810 06:40:48.187053 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:48.164828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:45303 (local address 127.31.59.254:40613)
0810 06:40:48.165419 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:48.165437 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:48.166159 (+ 722us) server_negotiation.cc:407] Connection header received
0810 06:40:48.168752 (+ 2593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:48.168764 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:48.169066 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:48.169362 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:48.171196 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:48.172410 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:48.174466 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:48.175089 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:48.175234 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:48.179088 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:48.179097 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:48.179115 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:48.179174 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:48.182573 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:48.183374 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:48.183382 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:48.183390 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:48.183590 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:48.183985 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:48.183990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:48.183994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:48.184610 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:48.184793 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:48.185510 (+ 717us) server_negotiation.cc:299] Negotiation successful
0810 06:40:48.185895 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":213,"threads_started":1}
I20260810 06:40:48.190073 24133 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40613
I20260810 06:40:48.190429 24133 heartbeater.cc:464] Registering TS with master...
I20260810 06:40:48.191385 24133 heartbeater.cc:511] Master 127.31.59.254:40613 requested a full tablet report, sending...
I20260810 06:40:48.194236 23843 ts_manager.cc:194] Registered new tserver with Master: f477755e860340688d4f8322b3af8afb (127.31.59.193:44425)
I20260810 06:40:48.196661 23843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:45303
W20260810 06:40:48.498878 24139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:48.499152 24139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:48.499249 24139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:48.506366 24139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:48.506469 24139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:48.506588 24139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:40:48.516166 24139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:40901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=46329
--tserver_master_addrs=127.31.59.254:40613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:48.517431 24139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:48.518774 24139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:48.531544 24157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:48.531603 24159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:48.531767 24156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:48.532388 24139 server_base.cc:1034] running on GCE node
I20260810 06:40:48.532968 24139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:48.533886 24139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:48.535079 24139 hybrid_clock.cc:648] HybridClock initialized: now 1786344048535023 us; error 48 us; skew 500 ppm
I20260810 06:40:48.535403 24139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:48 dist-test-slave-4tdj krb5kdc[22922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344048, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:48.538578 24139 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:40:48.540062 24139 webserver.cc:460] Webserver started at http://127.31.59.194:46329/ using document root <none> and password file <none>
I20260810 06:40:48.540735 24139 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:48.540867 24139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:48.546240 24139 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:40:48.549085 24169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:48.550472 24139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:40:48.550652 24139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "6b16ba59652c4075a91b93c5f7347cf8"
format_stamp: "Formatted at 2026-08-10 06:40:45 on dist-test-slave-4tdj"
server_key: "4f861bdd3f3f77277202ca850b1e14c1"
server_key_iv: "68fb9931f1a4c7eb74f5692e88dd1e7c"
server_key_version: "encryptionkey@0"
I20260810 06:40:48.551585 24139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:48.594096 24139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:48.598039 24139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:48.598330 24139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:48.599361 24139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:40:48.600713 24139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:40:48.600817 24139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:48.600920 24139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:40:48.600991 24139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:48.631556 24139 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:40901
I20260810 06:40:48.631625 24359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:40901 every 8 connection(s)
I20260810 06:40:48.632956 24139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:40:48.637259 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 24139
I20260810 06:40:48.637988 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:40593
--local_ip_for_outbound_sockets=127.31.59.195
--tserver_master_addrs=127.31.59.254:40613
--webserver_port=34973
--webserver_interface=127.31.59.195
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:40:48 dist-test-slave-4tdj krb5kdc[22922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344048, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:40:48.655169 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:48.635398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:48515 (local address 127.31.59.254:40613)
0810 06:40:48.635607 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:48.635613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:48.636749 (+ 1136us) server_negotiation.cc:407] Connection header received
0810 06:40:48.639216 (+ 2467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:48.639220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:48.639295 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:48.639427 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:48.641283 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:48.642005 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:48.644111 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:48.644600 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:48.644708 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:48.648634 (+ 3926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:48.648642 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:48.648646 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:48.648691 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:48.651866 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:48.652582 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:48.652588 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:48.652591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:48.652750 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:48.653208 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:48.653214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:48.653217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:48.653592 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:48.653746 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:48.654452 (+ 706us) server_negotiation.cc:299] Negotiation successful
0810 06:40:48.654702 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:40:48.655831 24364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:48.635907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:48515)
0810 06:40:48.636607 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:48.636643 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:48.638895 (+ 2252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:48.639685 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:48.639695 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:48.640357 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:48.641097 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:48.641120 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:48.642171 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:48.642175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:48.643932 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:48.643947 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:48.644823 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:48.644829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:48.645102 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:48.645812 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:48.645839 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:48.648393 (+ 2554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:48.652021 (+ 3628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:48.652028 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:48.652039 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:48.652442 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:48.652867 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:48.652870 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:48.652872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:48.653073 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:48.653718 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:48.653723 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:48.654022 (+ 299us) client_negotiation.cc:769] Sending connection context
0810 06:40:48.654238 (+ 216us) client_negotiation.cc:240] Negotiation successful
0810 06:40:48.654668 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":254,"threads_started":1}
I20260810 06:40:48.657264 24360 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40613
I20260810 06:40:48.657615 24360 heartbeater.cc:464] Registering TS with master...
I20260810 06:40:48.658595 24360 heartbeater.cc:511] Master 127.31.59.254:40613 requested a full tablet report, sending...
I20260810 06:40:48.660199 23843 ts_manager.cc:194] Registered new tserver with Master: 6b16ba59652c4075a91b93c5f7347cf8 (127.31.59.194:40901)
I20260810 06:40:48.661022 23843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:48515
W20260810 06:40:48.966575 24368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:48.966837 24368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:48.966977 24368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:48.973760 24368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:48.973867 24368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:48.974021 24368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:40:48.983156 24368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:40593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=34973
--tserver_master_addrs=127.31.59.254:40613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:48.984311 24368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:48.985570 24368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:48.998140 24374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:48.998175 24377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:48.998198 24379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:48.998828 24368 server_base.cc:1034] running on GCE node
I20260810 06:40:48.999372 24368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:49.000257 24368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:49.001441 24368 hybrid_clock.cc:648] HybridClock initialized: now 1786344049001395 us; error 45 us; skew 500 ppm
I20260810 06:40:49.001724 24368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:48 dist-test-slave-4tdj krb5kdc[22922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344048, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:49.004649 24368 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:40:49.005991 24368 webserver.cc:460] Webserver started at http://127.31.59.195:34973/ using document root <none> and password file <none>
I20260810 06:40:49.006680 24368 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:49.006790 24368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:49.011761 24368 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260810 06:40:49.014341 24389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:49.015684 24368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:40:49.015830 24368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "1638d375d1d747c296d64d0f08ac66b9"
format_stamp: "Formatted at 2026-08-10 06:40:46 on dist-test-slave-4tdj"
server_key: "2ee0eb3099216307e176a6a3bc397449"
server_key_iv: "a6c1be8a000be2362349c3be24585c41"
server_key_version: "encryptionkey@0"
I20260810 06:40:49.016742 24368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:49.040151 24368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:49.043941 24368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:49.044155 24368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:49.045035 24368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:40:49.046340 24368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:40:49.046429 24368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:49.046509 24368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:40:49.046569 24368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:49.078186 24368 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:40593
I20260810 06:40:49.078263 24583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:40593 every 8 connection(s)
I20260810 06:40:49.079675 24368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:40:49.087656 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 24368
Aug 10 06:40:49 dist-test-slave-4tdj krb5kdc[22922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344048, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:40:49.100323 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:49.082192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:55829 (local address 127.31.59.254:40613)
0810 06:40:49.082396 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:49.082401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:49.083573 (+ 1172us) server_negotiation.cc:407] Connection header received
0810 06:40:49.085360 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:49.085367 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:49.085448 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:49.085535 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:49.087013 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.087725 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.089661 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.090406 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.090532 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:49.094573 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:49.094580 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:49.094585 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:49.094623 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:49.097300 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:49.097975 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:49.097979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:49.097982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:49.098131 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:49.098620 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:49.098624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:49.098626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:49.098876 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:49.098986 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:49.099867 (+ 881us) server_negotiation.cc:299] Negotiation successful
0810 06:40:49.100050 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":51}
I20260810 06:40:49.101536 24588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:49.082711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:55829)
0810 06:40:49.083410 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:49.083446 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:49.085083 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:49.085779 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:49.085786 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:49.086265 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:49.086847 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:49.086864 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.087891 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.087897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:49.089503 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:49.089528 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.090626 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.090632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:49.090826 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:49.091458 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:49.091485 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:49.094323 (+ 2838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:49.097426 (+ 3103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:49.097435 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:49.097450 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:49.097845 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:49.098236 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:49.098241 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:49.098245 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:49.098497 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:49.099005 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:49.099013 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:49.099419 (+ 406us) client_negotiation.cc:769] Sending connection context
0810 06:40:49.099717 (+ 298us) client_negotiation.cc:240] Negotiation successful
0810 06:40:49.100233 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":276,"threads_started":1}
I20260810 06:40:49.102900 24584 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40613
I20260810 06:40:49.103199 24584 heartbeater.cc:464] Registering TS with master...
I20260810 06:40:49.103859 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:49.088703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40382 (local address 127.31.59.254:40613)
0810 06:40:49.089185 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:49.089191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:49.089496 (+ 305us) server_negotiation.cc:407] Connection header received
0810 06:40:49.089640 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:49.089645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:49.089707 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:49.089800 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:49.091590 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.092734 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.094405 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.095076 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.095197 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:49.097979 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:49.097984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:49.097987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:49.098037 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:49.101237 (+ 3200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:49.101949 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:49.101954 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:49.101957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:49.102107 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:49.102523 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:49.102528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:49.102532 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:49.102780 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:49.103000 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:49.103303 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 06:40:49.103497 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":361,"spinlock_wait_cycles":1408,"thread_start_us":238,"threads_started":1}
I20260810 06:40:49.103963 24584 heartbeater.cc:511] Master 127.31.59.254:40613 requested a full tablet report, sending...
I20260810 06:40:49.105389 23843 ts_manager.cc:194] Registered new tserver with Master: 1638d375d1d747c296d64d0f08ac66b9 (127.31.59.195:40593)
I20260810 06:40:49.106319 23843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:55829
I20260810 06:40:49.106384 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:40:49.112663 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:49.107336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40386 (local address 127.31.59.254:40613)
0810 06:40:49.107520 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:49.107525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:49.107856 (+ 331us) server_negotiation.cc:407] Connection header received
0810 06:40:49.108008 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:49.108012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:49.108156 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:49.108304 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:40:49.108756 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.109392 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.110828 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.111251 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.111338 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:49.111809 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:40:49.112219 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:40:49.112329 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:49.112416 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 06:40:49.112471 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:40:49.113473 23843 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:40386:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:40:49.120625 23843 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:40:49.154618 24606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:49.132230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:40901 (local address 127.0.0.1:48836)
0810 06:40:49.141394 (+ 9164us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:49.141439 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:49.141632 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:49.146691 (+ 5059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:49.146696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:49.146719 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:40:49.147103 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:49.147115 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.148733 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.148739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:49.151300 (+ 2561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:49.151312 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.153468 (+ 2156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.153475 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:49.153715 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:49.153822 (+ 107us) client_negotiation.cc:769] Sending connection context
0810 06:40:49.153942 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 06:40:49.154150 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8911,"mutex_wait_us":91,"thread_start_us":13667,"threads_started":1}
I20260810 06:40:49.154644 24601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:49.132515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48836 (local address 127.31.59.194:40901)
0810 06:40:49.145857 (+ 13342us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:49.145866 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:49.145900 (+ 34us) server_negotiation.cc:407] Connection header received
0810 06:40:49.145994 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:49.146001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:49.146230 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:49.146432 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:40:49.147284 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.148553 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.151444 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.152930 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.153070 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:49.153287 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:49.154092 (+ 805us) server_negotiation.cc:299] Negotiation successful
0810 06:40:49.154268 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13182,"thread_start_us":13066,"threads_started":1}
I20260810 06:40:49.155241 24600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:49.132408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:44425 (local address 127.0.0.1:53368)
0810 06:40:49.144308 (+ 11900us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:49.144329 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:49.144470 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:49.147248 (+ 2778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:49.147251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:49.147270 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:40:49.147500 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:49.147506 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.149268 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.149273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:49.151379 (+ 2106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:49.151393 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.153453 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.153460 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:49.153730 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:49.153821 (+ 91us) client_negotiation.cc:769] Sending connection context
0810 06:40:49.153942 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 06:40:49.154150 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11762,"mutex_wait_us":71,"thread_start_us":11614,"threads_started":1}
I20260810 06:40:49.155457 24603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:49.132791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53368 (local address 127.31.59.193:44425)
0810 06:40:49.143634 (+ 10843us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:49.143645 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:49.144622 (+ 977us) server_negotiation.cc:407] Connection header received
0810 06:40:49.144753 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:49.144763 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:49.145037 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:49.145259 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:40:49.147660 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.149099 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.151542 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.152930 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.153055 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:49.153254 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:49.154312 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0810 06:40:49.154477 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10630,"thread_start_us":10847,"threads_started":1}
I20260810 06:40:49.155777 24608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:49.136649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45628 (local address 127.31.59.195:40593)
0810 06:40:49.148098 (+ 11449us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:49.148107 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:49.148137 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:40:49.148225 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:49.148233 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:49.148436 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:49.148636 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:40:49.149427 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.151071 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.152715 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.153996 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.154129 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:49.154371 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:49.155071 (+ 700us) server_negotiation.cc:299] Negotiation successful
0810 06:40:49.155379 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11277,"thread_start_us":11110,"threads_started":1}
I20260810 06:40:49.156590 24599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:49.135985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:40593 (local address 127.0.0.1:45628)
0810 06:40:49.146175 (+ 10190us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:49.146193 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:49.146324 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:49.148645 (+ 2321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:49.148650 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:49.148676 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:40:49.149178 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:49.149191 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.151211 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.151216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:49.152564 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:49.152576 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.154532 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.154539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:49.154713 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:49.154743 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 06:40:49.156038 (+ 1295us) client_negotiation.cc:240] Negotiation successful
0810 06:40:49.156168 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10067,"mutex_wait_us":28,"thread_start_us":5013,"threads_started":1}
I20260810 06:40:49.158854 24027 tablet_service.cc:1467] Processing CreateTablet for tablet 1edf7dd1fde34ec698567628bbab2eef (DEFAULT_TABLE table=test-table [id=3663f9a60f1f47f285d7f4bb3ff544c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:40:49.158944 24252 tablet_service.cc:1467] Processing CreateTablet for tablet 1edf7dd1fde34ec698567628bbab2eef (DEFAULT_TABLE table=test-table [id=3663f9a60f1f47f285d7f4bb3ff544c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:40:49.160034 24027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1edf7dd1fde34ec698567628bbab2eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:49.160243 24252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1edf7dd1fde34ec698567628bbab2eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:49.160100 24484 tablet_service.cc:1467] Processing CreateTablet for tablet 1edf7dd1fde34ec698567628bbab2eef (DEFAULT_TABLE table=test-table [id=3663f9a60f1f47f285d7f4bb3ff544c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:40:49.161355 24484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1edf7dd1fde34ec698567628bbab2eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:49.174623 24609 tablet_bootstrap.cc:492] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb: Bootstrap starting.
I20260810 06:40:49.177901 24609 tablet_bootstrap.cc:654] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:49.179036 24613 tablet_bootstrap.cc:492] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9: Bootstrap starting.
I20260810 06:40:49.179119 24609 log.cc:824] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:49.180403 24612 tablet_bootstrap.cc:492] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8: Bootstrap starting.
I20260810 06:40:49.182531 24613 tablet_bootstrap.cc:654] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:49.183269 24612 tablet_bootstrap.cc:654] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:49.183741 24613 log.cc:824] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:49.183789 24609 tablet_bootstrap.cc:492] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb: No bootstrap required, opened a new log
I20260810 06:40:49.184043 24609 ts_tablet_manager.cc:1397] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 06:40:49.184414 24612 log.cc:824] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:49.188562 24613 tablet_bootstrap.cc:492] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9: No bootstrap required, opened a new log
I20260810 06:40:49.188750 24612 tablet_bootstrap.cc:492] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8: No bootstrap required, opened a new log
I20260810 06:40:49.188796 24613 ts_tablet_manager.cc:1397] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 06:40:49.189008 24612 ts_tablet_manager.cc:1397] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260810 06:40:49.194054 24609 raft_consensus.cc:348] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } } peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } }
I20260810 06:40:49.194518 24609 raft_consensus.cc:374] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:49.194609 24609 raft_consensus.cc:729] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f477755e860340688d4f8322b3af8afb, State: Initialized, Role: FOLLOWER
I20260810 06:40:49.195267 24609 consensus_queue.cc:260] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } } peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } }
I20260810 06:40:49.197340 24133 heartbeater.cc:503] Master 127.31.59.254:40613 was elected leader, sending a full tablet report...
I20260810 06:40:49.197961 24609 ts_tablet_manager.cc:1428] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260810 06:40:49.198226 24613 raft_consensus.cc:348] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } } peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } }
I20260810 06:40:49.198642 24613 raft_consensus.cc:374] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:49.198747 24613 raft_consensus.cc:729] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1638d375d1d747c296d64d0f08ac66b9, State: Initialized, Role: FOLLOWER
I20260810 06:40:49.198663 24612 raft_consensus.cc:348] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } } peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } }
I20260810 06:40:49.199033 24612 raft_consensus.cc:374] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:49.199131 24612 raft_consensus.cc:729] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b16ba59652c4075a91b93c5f7347cf8, State: Initialized, Role: FOLLOWER
I20260810 06:40:49.199319 24613 consensus_queue.cc:260] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } } peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } }
I20260810 06:40:49.199640 24612 consensus_queue.cc:260] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } } peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } }
I20260810 06:40:49.200455 24584 heartbeater.cc:503] Master 127.31.59.254:40613 was elected leader, sending a full tablet report...
I20260810 06:40:49.201112 24613 ts_tablet_manager.cc:1428] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260810 06:40:49.202102 24360 heartbeater.cc:503] Master 127.31.59.254:40613 was elected leader, sending a full tablet report...
I20260810 06:40:49.202602 24612 ts_tablet_manager.cc:1428] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8: Time spent starting tablet: real 0.013s user 0.010s sys 0.005s
I20260810 06:40:49.243909 24621 raft_consensus.cc:482] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:40:49.244206 24621 raft_consensus.cc:504] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } } peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } }
I20260810 06:40:49.245945 24621 leader_election.cc:288] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b16ba59652c4075a91b93c5f7347cf8 (127.31.59.194:40901), f477755e860340688d4f8322b3af8afb (127.31.59.193:44425)
I20260810 06:40:49.252068 24601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:49.246343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:51585 (local address 127.31.59.194:40901)
0810 06:40:49.246562 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:49.246568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:49.246595 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:40:49.246788 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:49.246794 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:49.246872 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:49.246953 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:40:49.247502 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.248212 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.250354 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.251127 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.251278 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:49.251353 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:49.251697 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 06:40:49.251833 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:40:49.252089 24588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:49.246157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:40901 (local address 127.31.59.195:51585)
0810 06:40:49.246430 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:49.246451 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:49.246573 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:49.247126 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:49.247129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:49.247156 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:40:49.247367 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:49.247379 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.248341 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.248345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:49.250187 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:49.250198 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.251273 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.251279 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:49.251522 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:49.251555 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:40:49.251641 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 06:40:49.251765 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":4,"spinlock_wait_cycles":33536}
I20260810 06:40:49.253266 24288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1edf7dd1fde34ec698567628bbab2eef" candidate_uuid: "1638d375d1d747c296d64d0f08ac66b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b16ba59652c4075a91b93c5f7347cf8" is_pre_election: true
I20260810 06:40:49.253501 24624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:49.246282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:44425 (local address 127.31.59.195:41839)
0810 06:40:49.247079 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:49.247093 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:49.247200 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:49.248357 (+ 1157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:49.248360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:49.248385 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:40:49.248725 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:49.248733 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.250059 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.250063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:49.251484 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:49.251496 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.252935 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.252939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:49.253067 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:49.253091 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 06:40:49.253163 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 06:40:49.253243 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":2,"thread_start_us":438,"threads_started":1}
I20260810 06:40:49.253575 24603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:49.246752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:41839 (local address 127.31.59.193:44425)
0810 06:40:49.246947 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:49.246952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:49.247194 (+ 242us) server_negotiation.cc:407] Connection header received
0810 06:40:49.247311 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:49.247316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:49.247387 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:49.247474 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:40:49.248893 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.249916 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.251664 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.252771 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.252930 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:49.252996 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:49.253213 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 06:40:49.253341 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:40:49.253860 24288 raft_consensus.cc:2454] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1638d375d1d747c296d64d0f08ac66b9 in term 0.
I20260810 06:40:49.254601 24391 leader_election.cc:302] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1638d375d1d747c296d64d0f08ac66b9, 6b16ba59652c4075a91b93c5f7347cf8; no voters:
I20260810 06:40:49.254644 24063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1edf7dd1fde34ec698567628bbab2eef" candidate_uuid: "1638d375d1d747c296d64d0f08ac66b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f477755e860340688d4f8322b3af8afb" is_pre_election: true
I20260810 06:40:49.254953 24621 raft_consensus.cc:2781] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:40:49.255087 24621 raft_consensus.cc:482] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:40:49.255185 24621 raft_consensus.cc:3037] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:49.255128 24063 raft_consensus.cc:2454] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1638d375d1d747c296d64d0f08ac66b9 in term 0.
I20260810 06:40:49.257470 24621 raft_consensus.cc:504] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } } peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } }
I20260810 06:40:49.258057 24621 leader_election.cc:288] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [CANDIDATE]: Term 1 election: Requested vote from peers 6b16ba59652c4075a91b93c5f7347cf8 (127.31.59.194:40901), f477755e860340688d4f8322b3af8afb (127.31.59.193:44425)
I20260810 06:40:49.258436 24288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1edf7dd1fde34ec698567628bbab2eef" candidate_uuid: "1638d375d1d747c296d64d0f08ac66b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b16ba59652c4075a91b93c5f7347cf8"
I20260810 06:40:49.258554 24063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1edf7dd1fde34ec698567628bbab2eef" candidate_uuid: "1638d375d1d747c296d64d0f08ac66b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f477755e860340688d4f8322b3af8afb"
I20260810 06:40:49.258651 24288 raft_consensus.cc:3037] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:49.258738 24063 raft_consensus.cc:3037] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:49.261734 24063 raft_consensus.cc:2454] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1638d375d1d747c296d64d0f08ac66b9 in term 1.
I20260810 06:40:49.261734 24288 raft_consensus.cc:2454] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1638d375d1d747c296d64d0f08ac66b9 in term 1.
I20260810 06:40:49.262107 24391 leader_election.cc:302] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1638d375d1d747c296d64d0f08ac66b9, 6b16ba59652c4075a91b93c5f7347cf8, f477755e860340688d4f8322b3af8afb; no voters:
I20260810 06:40:49.262420 24621 raft_consensus.cc:2781] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:40:49.262814 24621 raft_consensus.cc:686] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [term 1 LEADER]: Becoming Leader. State: Replica: 1638d375d1d747c296d64d0f08ac66b9, State: Running, Role: LEADER
I20260810 06:40:49.263240 24621 consensus_queue.cc:237] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } } peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } }
I20260810 06:40:49.267477 23843 catalog_manager.cc:5495] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1638d375d1d747c296d64d0f08ac66b9 (127.31.59.195). New cstate: current_term: 1 leader_uuid: "1638d375d1d747c296d64d0f08ac66b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1638d375d1d747c296d64d0f08ac66b9" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 40593 } health_report { overall_health: HEALTHY } } }
I20260810 06:40:49.287290 24608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:49.282128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45636 (local address 127.31.59.195:40593)
0810 06:40:49.282350 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:49.282355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:49.282374 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:40:49.282512 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:49.282517 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:49.282606 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:49.282730 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:40:49.283111 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.283789 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.285251 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.285632 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.285744 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:49.286204 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:40:49.286733 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:40:49.286859 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:49.286968 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 06:40:49.287054 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:40:49.295295 24063 raft_consensus.cc:1261] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb [term 1 FOLLOWER]: Refusing update from remote peer 1638d375d1d747c296d64d0f08ac66b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:40:49.295292 24288 raft_consensus.cc:1261] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8 [term 1 FOLLOWER]: Refusing update from remote peer 1638d375d1d747c296d64d0f08ac66b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:40:49.296352 24621 consensus_queue.cc:1035] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b16ba59652c4075a91b93c5f7347cf8" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:40:49.296651 24626 consensus_queue.cc:1035] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f477755e860340688d4f8322b3af8afb" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 44425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:40:49.322111 23842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:40386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:40:49.322402 23842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:40386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:40:49.325748 23842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d: Sending DeleteTablet for 3 replicas of tablet 1edf7dd1fde34ec698567628bbab2eef
I20260810 06:40:49.326858 24252 tablet_service.cc:1514] Processing DeleteTablet for tablet 1edf7dd1fde34ec698567628bbab2eef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:40:49 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:48836
I20260810 06:40:49.326857 24027 tablet_service.cc:1514] Processing DeleteTablet for tablet 1edf7dd1fde34ec698567628bbab2eef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:40:49 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:53368
I20260810 06:40:49.327755 24655 tablet_replica.cc:330] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8: stopping tablet replica
I20260810 06:40:49.328197 24655 raft_consensus.cc:2229] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:40:49.328347 24484 tablet_service.cc:1514] Processing DeleteTablet for tablet 1edf7dd1fde34ec698567628bbab2eef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:40:49 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:45628
I20260810 06:40:49.328811 24655 raft_consensus.cc:2258] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:40:49.329394 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 23894
I20260810 06:40:49.331213 24655 ts_tablet_manager.cc:1905] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:40:49.338140 24655 ts_tablet_manager.cc:1918] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:40:49.338747 24657 tablet_replica.cc:330] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb: stopping tablet replica
I20260810 06:40:49.339155 24657 raft_consensus.cc:2229] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:40:49.339650 24657 raft_consensus.cc:2258] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:40:49.340040 24655 log.cc:1196] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/1edf7dd1fde34ec698567628bbab2eef
I20260810 06:40:49.340615 24655 ts_tablet_manager.cc:1939] T 1edf7dd1fde34ec698567628bbab2eef P 6b16ba59652c4075a91b93c5f7347cf8: Deleting consensus metadata
I20260810 06:40:49.342003 24657 ts_tablet_manager.cc:1905] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:40:49.342417 23819 catalog_manager.cc:4841] TS 6b16ba59652c4075a91b93c5f7347cf8 (127.31.59.194:40901): tablet 1edf7dd1fde34ec698567628bbab2eef (table test-table [id=3663f9a60f1f47f285d7f4bb3ff544c6]) successfully deleted
I20260810 06:40:49.350111 24657 ts_tablet_manager.cc:1918] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:40:49.350275 24657 log.cc:1196] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/1edf7dd1fde34ec698567628bbab2eef
I20260810 06:40:49.350919 24657 ts_tablet_manager.cc:1939] T 1edf7dd1fde34ec698567628bbab2eef P f477755e860340688d4f8322b3af8afb: Deleting consensus metadata
I20260810 06:40:49.352262 24603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:49.330594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53380 (local address 127.31.59.193:44425)
0810 06:40:49.331212 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:49.331218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:49.331236 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:40:49.331293 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:49.331299 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:49.331395 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:49.331489 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:49.333497 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.334801 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.336952 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.337683 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.337802 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:49.344929 (+ 7127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:49.344944 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:49.344961 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:49.345002 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:49.348990 (+ 3988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:49.349829 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:49.349838 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:49.349842 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:49.349990 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:49.350441 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:49.350447 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:49.350450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:49.351058 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:49.351358 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:49.351666 (+ 308us) server_negotiation.cc:299] Negotiation successful
0810 06:40:49.351898 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486}
I20260810 06:40:49.352725 23820 catalog_manager.cc:4841] TS f477755e860340688d4f8322b3af8afb (127.31.59.193:44425): tablet 1edf7dd1fde34ec698567628bbab2eef (table test-table [id=3663f9a60f1f47f285d7f4bb3ff544c6]) successfully deleted
I20260810 06:40:49.353158 24660 tablet_replica.cc:330] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9: stopping tablet replica
I20260810 06:40:49.353202 24127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:49.353585 24660 raft_consensus.cc:2229] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:40:49.354250 24660 raft_consensus.cc:2258] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:40:49.356494 24660 ts_tablet_manager.cc:1905] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:40:49.363688 24660 ts_tablet_manager.cc:1918] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:40:49.363855 24660 log.cc:1196] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/1edf7dd1fde34ec698567628bbab2eef
I20260810 06:40:49.364431 24660 ts_tablet_manager.cc:1939] T 1edf7dd1fde34ec698567628bbab2eef P 1638d375d1d747c296d64d0f08ac66b9: Deleting consensus metadata
I20260810 06:40:49.366102 23821 catalog_manager.cc:4841] TS 1638d375d1d747c296d64d0f08ac66b9 (127.31.59.195:40593): tablet 1edf7dd1fde34ec698567628bbab2eef (table test-table [id=3663f9a60f1f47f285d7f4bb3ff544c6]) successfully deleted
I20260810 06:40:49.440378 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 23894
I20260810 06:40:49.468840 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 24139
I20260810 06:40:49.484498 24601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:49.469670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48846 (local address 127.31.59.194:40901)
0810 06:40:49.469836 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:49.469840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:49.469988 (+ 148us) server_negotiation.cc:407] Connection header received
0810 06:40:49.470144 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:49.470150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:49.470235 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:49.470455 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:49.471883 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.473009 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.474332 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.474972 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.475096 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:49.477752 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:49.477772 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:49.477793 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:49.477844 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:49.481545 (+ 3701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:49.482263 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:49.482274 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:49.482278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:49.482453 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:49.482900 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:49.482905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:49.482908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:49.483363 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:49.483669 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:49.483902 (+ 233us) server_negotiation.cc:299] Negotiation successful
0810 06:40:49.484128 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:40:49.485333 24352 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:49.572006 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 24139
I20260810 06:40:49.597749 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 24368
I20260810 06:40:49.613250 24608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:49.598589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45650 (local address 127.31.59.195:40593)
0810 06:40:49.598812 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:49.598819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:49.598880 (+ 61us) server_negotiation.cc:407] Connection header received
0810 06:40:49.599047 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:49.599053 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:49.599139 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:49.599263 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:49.600539 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.601795 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.603234 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:49.603940 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:49.604083 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:49.606613 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:49.606628 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:49.606653 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:49.606698 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:49.610426 (+ 3728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:49.611079 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:49.611088 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:49.611092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:49.611302 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:49.611657 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:49.611662 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:49.611665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:49.612092 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:49.612421 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:49.612609 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 06:40:49.612819 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:40:49.614043 24579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:49.746703 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 24368
I20260810 06:40:49.776006 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 23793
I20260810 06:40:49.777197 23883 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:40:49.887490 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 23793
I20260810 06:40:49.915107 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40613
--webserver_interface=127.31.59.254
--webserver_port=45513
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:40613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:50.243808 24669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:50.244060 24669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:50.244175 24669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:50.251081 24669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:50.251171 24669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:50.251242 24669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:50.251317 24669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:50.251376 24669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:50.260573 24669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:40613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=45513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:50.261780 24669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:50.263032 24669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:50.273839 24683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:50.273917 24681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:50.274256 24677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:50.274590 24669 server_base.cc:1034] running on GCE node
I20260810 06:40:50.275269 24669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:50.276533 24669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:50.277722 24669 hybrid_clock.cc:648] HybridClock initialized: now 1786344050277665 us; error 51 us; skew 500 ppm
I20260810 06:40:50.278017 24669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:50 dist-test-slave-4tdj krb5kdc[22922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344050, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:50.280882 24669 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:40:50.282219 24669 webserver.cc:460] Webserver started at http://127.31.59.254:45513/ using document root <none> and password file <none>
I20260810 06:40:50.282872 24669 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:50.282987 24669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:50.288004 24669 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 06:40:50.290645 24693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:50.291962 24669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:40:50.292135 24669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "625f6510ab8e463dbed5383ee32adc0d"
format_stamp: "Formatted at 2026-08-10 06:40:44 on dist-test-slave-4tdj"
server_key: "3d86bbfb462906dfc19ec89d043fdabe"
server_key_iv: "c86dcc0a94220e27d3da9b04daae4de9"
server_key_version: "encryptionkey@0"
I20260810 06:40:50.293090 24669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:50.308879 24669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:50.312597 24669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:50.312800 24669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:50.330080 24669 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:40613
I20260810 06:40:50.330087 24774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:40613 every 8 connection(s)
I20260810 06:40:50.331483 24669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:40:50.337977 24775 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:40:50.339027 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 24669
I20260810 06:40:50.340034 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:44425
--local_ip_for_outbound_sockets=127.31.59.193
--tserver_master_addrs=127.31.59.254:40613
--webserver_port=36821
--webserver_interface=127.31.59.193
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:40:50.342267 24775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d: Bootstrap starting.
I20260810 06:40:50.364048 24775 log.cc:824] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:50.379962 24775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:40:50.380465 24775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d: Bootstrap complete.
I20260810 06:40:50.389178 24775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625f6510ab8e463dbed5383ee32adc0d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40613 } }
I20260810 06:40:50.390008 24775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 625f6510ab8e463dbed5383ee32adc0d, State: Initialized, Role: FOLLOWER
I20260810 06:40:50.390606 24775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625f6510ab8e463dbed5383ee32adc0d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40613 } }
I20260810 06:40:50.390825 24775 raft_consensus.cc:388] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:40:50.390928 24775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:40:50.391100 24775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 2 FOLLOWER]: Advancing to term 3
I20260810 06:40:50.395002 24775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625f6510ab8e463dbed5383ee32adc0d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40613 } }
I20260810 06:40:50.395462 24775 leader_election.cc:302] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 625f6510ab8e463dbed5383ee32adc0d; no voters:
I20260810 06:40:50.396076 24775 leader_election.cc:288] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 06:40:50.396209 24781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 3 FOLLOWER]: Leader election won for term 3
I20260810 06:40:50.398173 24775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:50.403246 24781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [term 3 LEADER]: Becoming Leader. State: Replica: 625f6510ab8e463dbed5383ee32adc0d, State: Running, Role: LEADER
I20260810 06:40:50.403824 24781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625f6510ab8e463dbed5383ee32adc0d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40613 } }
I20260810 06:40:50.406265 24786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "625f6510ab8e463dbed5383ee32adc0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625f6510ab8e463dbed5383ee32adc0d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40613 } } }
I20260810 06:40:50.406636 24786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:50.407177 24787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 625f6510ab8e463dbed5383ee32adc0d. Latest consensus state: current_term: 3 leader_uuid: "625f6510ab8e463dbed5383ee32adc0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625f6510ab8e463dbed5383ee32adc0d" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40613 } } }
I20260810 06:40:50.407439 24787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 625f6510ab8e463dbed5383ee32adc0d [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:50.409405 24801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:40:50.429807 24801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:40:50.431361 24801 catalog_manager.cc:1259] Loaded cluster ID: d0f6080ccbc946f7a2421d05a9764806
I20260810 06:40:50.431466 24801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 06:40:50.433624 24801 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fae0e0ee9ad google::LogMessage::Fail() at ??:0
@ 0x7fae0e0f188a google::LogMessage::SendToLog() at ??:0
@ 0x7fae0e0ee4de google::LogMessage::Flush() at ??:0
@ 0x7fae0e0efa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fae23b21512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fae23b1e976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fae23ba6610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fae23ba6439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fae10d3a345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fae10d3d91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fae10d3d799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fae10d18eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fae13ff26db start_thread at ??:0
@ 0x7fae0cf8861f clone at ??:0
W20260810 06:40:50.701153 24779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:50.701449 24779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:50.701599 24779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:50.708940 24779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:50.709069 24779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:50.709223 24779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:40:50.719156 24779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:44425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=36821
--tserver_master_addrs=127.31.59.254:40613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:50.720428 24779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:50.721769 24779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:50.734877 24836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:50.734833 24839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:50.734901 24845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:50.735666 24779 server_base.cc:1034] running on GCE node
I20260810 06:40:50.736240 24779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:50.737164 24779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:50.738384 24779 hybrid_clock.cc:648] HybridClock initialized: now 1786344050738334 us; error 33 us; skew 500 ppm
I20260810 06:40:50.738688 24779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:50 dist-test-slave-4tdj krb5kdc[22922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344050, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:50.741608 24779 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:40:50.743005 24779 webserver.cc:460] Webserver started at http://127.31.59.193:36821/ using document root <none> and password file <none>
I20260810 06:40:50.743650 24779 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:50.743772 24779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:50.748955 24779 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 06:40:50.751641 24856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:50.752964 24779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:40:50.753127 24779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "f477755e860340688d4f8322b3af8afb"
format_stamp: "Formatted at 2026-08-10 06:40:45 on dist-test-slave-4tdj"
server_key: "a5a2a6a8cce2a90c5ae4d9240b978591"
server_key_iv: "74399654fcf2085d9c4149b5b839eb06"
server_key_version: "encryptionkey@0"
I20260810 06:40:50.754072 24779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:50.780431 24779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:50.784415 24779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:50.784693 24779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:50.785758 24779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:40:50.787119 24779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:40:50.787235 24779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:50.787336 24779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:40:50.787410 24779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:50.819067 24779 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:44425
I20260810 06:40:50.819132 25037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:44425 every 8 connection(s)
I20260810 06:40:50.820472 24779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:40:50.825477 25040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:50.823426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:45521)
0810 06:40:50.824165 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:50.824397 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":263,"threads_started":1}
W20260810 06:40:50.826227 25038 heartbeater.cc:644] Failed to heartbeat to 127.31.59.254:40613 (0 consecutive failures): Network error: Failed to ping master at 127.31.59.254:40613: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
I20260810 06:40:50.827066 25040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:50.826716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:44535)
0810 06:40:50.826863 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:50.826942 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 06:40:50.827895 25040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:50.827566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:39787)
0810 06:40:50.827692 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:50.827787 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 06:40:50.828145 25038 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:40:50.828289 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 24779
I20260810 06:40:50.828897 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:40901
--local_ip_for_outbound_sockets=127.31.59.194
--tserver_master_addrs=127.31.59.254:40613
--webserver_port=46329
--webserver_interface=127.31.59.194
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:51.158753 25042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:51.159010 25042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:51.159139 25042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:51.166221 25042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:51.166354 25042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:51.166522 25042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:40:51.175990 25042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:40901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=46329
--tserver_master_addrs=127.31.59.254:40613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:51.177229 25042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:51.178607 25042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:51.191129 25059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:51.191140 25052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:51.191375 25050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:51.192068 25042 server_base.cc:1034] running on GCE node
I20260810 06:40:51.192647 25042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:51.193536 25042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:51.194749 25042 hybrid_clock.cc:648] HybridClock initialized: now 1786344051194695 us; error 47 us; skew 500 ppm
I20260810 06:40:51.195048 25042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:51 dist-test-slave-4tdj krb5kdc[22922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344051, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:51.198050 25042 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:40:51.201684 25042 webserver.cc:460] Webserver started at http://127.31.59.194:46329/ using document root <none> and password file <none>
I20260810 06:40:51.202448 25042 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:51.202584 25042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:51.207923 25042 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:40:51.210551 25068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:51.211850 25042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:40:51.211993 25042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "6b16ba59652c4075a91b93c5f7347cf8"
format_stamp: "Formatted at 2026-08-10 06:40:45 on dist-test-slave-4tdj"
server_key: "4f861bdd3f3f77277202ca850b1e14c1"
server_key_iv: "68fb9931f1a4c7eb74f5692e88dd1e7c"
server_key_version: "encryptionkey@0"
I20260810 06:40:51.212987 25042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:51.230526 25042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:51.234496 25042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:51.234754 25042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:51.235754 25042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:40:51.237068 25042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:40:51.237160 25042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:51.237267 25042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:40:51.237331 25042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:51.268090 25042 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:40901
I20260810 06:40:51.268149 25231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:40901 every 8 connection(s)
I20260810 06:40:51.269532 25042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:40:51.273890 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 25042
I20260810 06:40:51.274423 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:51.272513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:46555)
0810 06:40:51.273244 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:51.273458 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":256,"threads_started":1}
I20260810 06:40:51.274580 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:40593
--local_ip_for_outbound_sockets=127.31.59.195
--tserver_master_addrs=127.31.59.254:40613
--webserver_port=34973
--webserver_interface=127.31.59.195
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:51.275094 25233 heartbeater.cc:644] Failed to heartbeat to 127.31.59.254:40613 (0 consecutive failures): Network error: Failed to ping master at 127.31.59.254:40613: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
I20260810 06:40:51.275869 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:51.275582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:39917)
0810 06:40:51.275708 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:51.275771 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 06:40:51.276628 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:51.276350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:37113)
0810 06:40:51.276460 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:51.276528 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 06:40:51.276830 25233 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:40:51.594637 25238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:51.594890 25238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:51.594991 25238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:51.601830 25238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:51.601922 25238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:51.602033 25238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:40:51.611197 25238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:40593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=34973
--tserver_master_addrs=127.31.59.254:40613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:40:51.612317 25238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:51.613631 25238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:51.625909 25248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:51.625895 25245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:51.625950 25246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:51.627044 25238 server_base.cc:1034] running on GCE node
I20260810 06:40:51.627581 25238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:51.628475 25238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:51.629689 25238 hybrid_clock.cc:648] HybridClock initialized: now 1786344051629642 us; error 34 us; skew 500 ppm
I20260810 06:40:51.629966 25238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:40:51 dist-test-slave-4tdj krb5kdc[22922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344051, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:51.632961 25238 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:40:51.634315 25238 webserver.cc:460] Webserver started at http://127.31.59.195:34973/ using document root <none> and password file <none>
I20260810 06:40:51.634953 25238 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:51.635072 25238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:51.640136 25238 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260810 06:40:51.642745 25255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:51.644016 25238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:40:51.644166 25238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "1638d375d1d747c296d64d0f08ac66b9"
format_stamp: "Formatted at 2026-08-10 06:40:46 on dist-test-slave-4tdj"
server_key: "2ee0eb3099216307e176a6a3bc397449"
server_key_iv: "a6c1be8a000be2362349c3be24585c41"
server_key_version: "encryptionkey@0"
I20260810 06:40:51.645084 25238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:51.667797 25238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:51.671769 25238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:51.672017 25238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:51.672919 25238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:40:51.674197 25238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:40:51.674319 25238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:51.674424 25238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:40:51.674495 25238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:51.704486 25238 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:40593
I20260810 06:40:51.704556 25439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:40593 every 8 connection(s)
I20260810 06:40:51.705879 25238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:40:51.709438 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 25238
I20260810 06:40:51.710794 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:51.708682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:58733)
0810 06:40:51.709506 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:51.709809 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":287,"threads_started":1}
W20260810 06:40:51.711475 25442 heartbeater.cc:644] Failed to heartbeat to 127.31.59.254:40613 (0 consecutive failures): Network error: Failed to ping master at 127.31.59.254:40613: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
I20260810 06:40:51.712180 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:51.711883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:57443)
0810 06:40:51.712011 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:51.712079 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 06:40:51.712970 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:51.712653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:46203)
0810 06:40:51.712789 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:51.712861 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 06:40:51.713253 25442 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:40:51.829362 25449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:51.828641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:57895)
0810 06:40:51.829058 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:51.829151 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":204,"threads_started":1}
I20260810 06:40:52.278108 25452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:52.277322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:34309)
0810 06:40:52.277769 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:52.277881 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":217,"threads_started":1}
I20260810 06:40:52.714622 25453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:52.713769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:60775)
0810 06:40:52.714269 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:52.714393 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":233,"threads_started":1}
I20260810 06:40:52.830920 25456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:52.830186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:51487)
0810 06:40:52.830682 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:52.830764 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":237,"threads_started":1}
I20260810 06:40:53.279630 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:53.278930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:39615)
0810 06:40:53.279386 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:53.279467 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":228,"threads_started":1}
I20260810 06:40:53.716251 25458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:53.715499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:46977)
0810 06:40:53.715975 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:53.716057 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":212,"threads_started":1}
I20260810 06:40:53.832407 25459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:53.831682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:42527)
0810 06:40:53.832142 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:53.832229 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":242,"threads_started":1}
I20260810 06:40:54.281158 25460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:54.280456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:56561)
0810 06:40:54.280899 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:54.280993 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":241,"threads_started":1}
I20260810 06:40:54.717783 25461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:54.717067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:58477)
0810 06:40:54.717475 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:54.717559 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":208,"threads_started":1}
I20260810 06:40:54.833976 25462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:54.833255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:56939)
0810 06:40:54.833721 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:54.833803 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":258,"threads_started":1}
I20260810 06:40:55.282799 25464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:55.282037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:54257)
0810 06:40:55.282522 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:55.282606 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":244,"threads_started":1}
I20260810 06:40:55.719360 25467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:55.718646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:46415)
0810 06:40:55.719085 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:55.719170 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":199,"threads_started":1}
I20260810 06:40:55.835489 25470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:55.834818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:41733)
0810 06:40:55.835253 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:55.835333 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":235,"threads_started":1}
I20260810 06:40:56.284308 25472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:56.283628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:56785)
0810 06:40:56.284062 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:56.284141 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":216,"threads_started":1}
I20260810 06:40:56.720912 25474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:56.720217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:42611)
0810 06:40:56.720670 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:56.720750 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":222,"threads_started":1}
I20260810 06:40:56.836985 25475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:56.836299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:56159)
0810 06:40:56.836746 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:56.836834 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":219,"threads_started":1}
I20260810 06:40:57.285790 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:57.285092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:54623)
0810 06:40:57.285549 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:57.285634 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":243,"threads_started":1}
I20260810 06:40:57.722541 25477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:57.721739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:42769)
0810 06:40:57.722230 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:57.722359 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":255,"threads_started":1}
I20260810 06:40:57.838721 25479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:57.837855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:59271)
0810 06:40:57.838391 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:57.838486 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":215,"threads_started":1}
I20260810 06:40:58.287474 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:58.286610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:38565)
0810 06:40:58.287106 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:58.287210 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":220,"threads_started":1}
I20260810 06:40:58.724315 25482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:58.723497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:53051)
0810 06:40:58.724016 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:58.724102 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":236,"threads_started":1}
I20260810 06:40:58.840343 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:58.839629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:57379)
0810 06:40:58.840073 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:58.840163 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":213,"threads_started":1}
I20260810 06:40:59.289062 25484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:59.288335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:56763)
0810 06:40:59.288805 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:59.288888 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":248,"threads_started":1}
I20260810 06:40:59.725965 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:59.725255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:43993)
0810 06:40:59.725724 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:59.725806 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":240,"threads_started":1}
I20260810 06:40:59.841902 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:59.841169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:58619)
0810 06:40:59.841625 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:59.841712 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":217,"threads_started":1}
I20260810 06:41:00.290711 25492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:00.289859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:43599)
0810 06:41:00.290407 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:00.290495 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":242,"threads_started":1}
I20260810 06:41:00.727643 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:00.726881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:52415)
0810 06:41:00.727379 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:00.727468 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":232,"threads_started":1}
I20260810 06:41:00.843485 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:00.842797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:45167)
0810 06:41:00.843249 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:00.843331 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":241,"threads_started":1}
I20260810 06:41:01.292371 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:01.291631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:39173)
0810 06:41:01.292072 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:01.292175 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":234,"threads_started":1}
I20260810 06:41:01.729316 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:01.728559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:41775)
0810 06:41:01.729057 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:01.729152 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":291,"threads_started":1}
I20260810 06:41:01.845036 25501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:01.844297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:54605)
0810 06:41:01.844761 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:01.844850 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":242,"threads_started":1}
I20260810 06:41:02.294044 25502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:02.293284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:43231)
0810 06:41:02.293760 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:02.293850 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":262,"threads_started":1}
I20260810 06:41:02.730981 25503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:02.730241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:56349)
0810 06:41:02.730705 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:02.730801 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":242,"threads_started":1}
I20260810 06:41:02.846647 25504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:02.845929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:52807)
0810 06:41:02.846404 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:02.846488 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":238,"threads_started":1}
I20260810 06:41:03.295583 25505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:03.294906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:42051)
0810 06:41:03.295349 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:03.295427 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":240,"threads_started":1}
I20260810 06:41:03.732614 25506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:03.731889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:51467)
0810 06:41:03.732363 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:03.732451 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":235,"threads_started":1}
I20260810 06:41:03.848369 25507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:03.847546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:44345)
0810 06:41:03.848004 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:03.848122 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":241,"threads_started":1}
I20260810 06:41:04.297344 25511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:04.296467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:47247)
0810 06:41:04.296987 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:04.297091 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":247,"threads_started":1}
I20260810 06:41:04.734285 25513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:04.733527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:59023)
0810 06:41:04.734001 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:04.734099 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":220,"threads_started":1}
I20260810 06:41:04.849936 25514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:04.849274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:39983)
0810 06:41:04.849683 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:04.849762 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":209,"threads_started":1}
I20260810 06:41:05.299106 25516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:05.298248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:53881)
0810 06:41:05.298788 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:05.298897 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":311,"threads_started":1}
I20260810 06:41:05.735978 25518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:05.735194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:33941)
0810 06:41:05.735681 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:05.735783 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":250,"threads_started":1}
I20260810 06:41:05.851598 25521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:05.850921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.193:36461)
0810 06:41:05.851325 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:05.851428 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":197,"threads_started":1}
I20260810 06:41:06.300971 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:06.300210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.194:56041)
0810 06:41:06.300656 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:06.300748 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":218,"threads_started":1}
I20260810 06:41:06.714881 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 24779
I20260810 06:41:06.732641 25523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:06.715924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45592 (local address 127.31.59.193:44425)
0810 06:41:06.716656 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:06.716675 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:06.716720 (+ 45us) server_negotiation.cc:407] Connection header received
0810 06:41:06.718503 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:06.718513 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:06.718947 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:06.719298 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:06.720755 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:06.721974 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:06.723238 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:06.723908 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:06.724111 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:06.726770 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:06.726777 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:06.726792 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:06.726833 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:06.729468 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:06.730093 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:06.730100 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:06.730106 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:06.730271 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:06.730669 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:06.730674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:06.730678 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:06.731197 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:06.731496 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:06.731942 (+ 446us) server_negotiation.cc:299] Negotiation successful
0810 06:41:06.732364 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":288,"threads_started":1}
I20260810 06:41:06.734025 25029 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:41:06.738139 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:06.737156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40613 (local address 127.31.59.195:45819)
0810 06:41:06.737746 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:06.737909 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:40613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":308,"threads_started":1}
I20260810 06:41:06.815878 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 24779
I20260810 06:41:06.838181 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 25042
I20260810 06:41:06.856338 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:06.839269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56314 (local address 127.31.59.194:40901)
0810 06:41:06.839841 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:06.839855 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:06.839893 (+ 38us) server_negotiation.cc:407] Connection header received
0810 06:41:06.841477 (+ 1584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:06.841487 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:06.841905 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:06.842211 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:06.843788 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:06.845091 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:06.846433 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:06.847094 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:06.847223 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:06.849993 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:06.850002 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:06.850016 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:06.850057 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:06.853246 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:06.853916 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:06.853923 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:06.853929 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:06.854142 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:06.854527 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:06.854530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:06.854532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:06.855064 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:06.855321 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:06.855640 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 06:41:06.856038 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":261,"threads_started":1}
I20260810 06:41:06.857654 25226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:41:06.939388 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 25042
I20260810 06:41:06.962478 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 25238
I20260810 06:41:06.980057 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:06.963469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49156 (local address 127.31.59.195:40593)
0810 06:41:06.964110 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:06.964125 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:06.964166 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:41:06.965775 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:06.965785 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:06.966219 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:06.966666 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:06.968071 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:06.969301 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:06.970572 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:06.971224 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:06.971401 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:06.973986 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:06.973994 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:06.974007 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:06.974049 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:06.976783 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:06.977508 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:06.977516 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:06.977523 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:06.977706 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:06.978063 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:06.978067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:06.978069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:06.978617 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:06.978932 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:06.979335 (+ 403us) server_negotiation.cc:299] Negotiation successful
0810 06:41:06.979750 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":299,"threads_started":1}
I20260810 06:41:06.981460 25431 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:41:07.066186 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 25238
2026-08-10T06:41:07Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24219 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:41:07 dist-test-slave-4tdj krb5kdc[25533](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:41:07 dist-test-slave-4tdj krb5kdc[25533](info): set up 2 sockets
Aug 10 06:41:07 dist-test-slave-4tdj krb5kdc[25533](info): commencing operation
krb5kdc: starting...
W20260810 06:41:08.515296 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.402s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:41:08 dist-test-slave-4tdj krb5kdc[25533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:41:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:41:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:41:08.959057 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:42153
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:42153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:41:09.508981 25557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:41:09.509452 25557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:41:09.509675 25557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:41:09.521121 25557 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 06:41:09.521281 25557 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 06:41:09.521411 25557 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 06:41:09.521550 25557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:41:09.521703 25557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:41:09.521876 25557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:41:09.521971 25557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:41:09.522102 25557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:41:09.538075 25557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:42153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:42153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:41:09.540278 25557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:41:09.542665 25557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:41:09.561497 25567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:09.561530 25563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:09.561456 25565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:41:09.562700 25557 server_base.cc:1034] running on GCE node
I20260810 06:41:09.563927 25557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:41:09.566355 25557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:41:09.567683 25557 hybrid_clock.cc:648] HybridClock initialized: now 1786344069567584 us; error 49 us; skew 500 ppm
I20260810 06:41:09.568235 25557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:41:09 dist-test-slave-4tdj krb5kdc[25533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344069, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:41:09.578816 25557 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:41:09.581097 25557 webserver.cc:460] Webserver started at http://127.31.59.254:43485/ using document root <none> and password file <none>
I20260810 06:41:09.582185 25557 fs_manager.cc:359] Metadata directory not provided
I20260810 06:41:09.582361 25557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:41:09.582820 25557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:41:09.587805 25557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "b187e4bc178e4af998b9cd688de15601"
format_stamp: "Formatted at 2026-08-10 06:41:09 on dist-test-slave-4tdj"
server_key: "1393d5f8b33cd7161f07558038fcbbd1"
server_key_iv: "bebe20a558133f48e3451e1e52e81900"
server_key_version: "encryptionkey@0"
I20260810 06:41:09.589130 25557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "b187e4bc178e4af998b9cd688de15601"
format_stamp: "Formatted at 2026-08-10 06:41:09 on dist-test-slave-4tdj"
server_key: "1393d5f8b33cd7161f07558038fcbbd1"
server_key_iv: "bebe20a558133f48e3451e1e52e81900"
server_key_version: "encryptionkey@0"
I20260810 06:41:09.597775 25557 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260810 06:41:09.603399 25577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:09.605607 25557 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260810 06:41:09.605841 25557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "b187e4bc178e4af998b9cd688de15601"
format_stamp: "Formatted at 2026-08-10 06:41:09 on dist-test-slave-4tdj"
server_key: "1393d5f8b33cd7161f07558038fcbbd1"
server_key_iv: "bebe20a558133f48e3451e1e52e81900"
server_key_version: "encryptionkey@0"
I20260810 06:41:09.606184 25557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:41:09.629595 25557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:41:09.636081 25557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:41:09.636484 25557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:41:09.665367 25557 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:42153
I20260810 06:41:09.665382 25653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:42153 every 8 connection(s)
I20260810 06:41:09.667665 25557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:41:09.668404 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 25557
I20260810 06:41:09.668713 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:41:09.670210 31983 external_mini_cluster.cc:1428] Setting key 39b9ffd29916fd3c352d7faa12d691fb
I20260810 06:41:09.675045 25654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:41:09 dist-test-slave-4tdj krb5kdc[25533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:41:09.694442 25654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: Bootstrap starting.
I20260810 06:41:09.698345 25654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: Neither blocks nor log segments found. Creating new log.
I20260810 06:41:09.699579 25654 log.cc:824] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: Log is configured to *not* fsync() on all Append() calls
I20260810 06:41:09.702870 25654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: No bootstrap required, opened a new log
I20260810 06:41:09.703203 25658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:09.673554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33852 (local address 127.31.59.254:42153)
0810 06:41:09.674896 (+ 1342us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:09.674935 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:09.675057 (+ 122us) server_negotiation.cc:407] Connection header received
0810 06:41:09.679529 (+ 4472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:09.679620 (+ 91us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:09.680506 (+ 886us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:09.681309 (+ 803us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:09.683611 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:09.687086 (+ 3475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:09.689218 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:09.690827 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:09.691004 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:09.695044 (+ 4040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:09.695054 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:09.695072 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:09.695121 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:09.698987 (+ 3866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:09.699728 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:09.699736 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:09.699743 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:09.699947 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:09.700358 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:09.700361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:09.700363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:09.700874 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:09.701133 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:09.701519 (+ 386us) server_negotiation.cc:299] Negotiation successful
0810 06:41:09.701939 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":981,"thread_start_us":480,"threads_started":1}
I20260810 06:41:09.710837 25654 raft_consensus.cc:348] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e4bc178e4af998b9cd688de15601" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 42153 } }
I20260810 06:41:09.711258 25654 raft_consensus.cc:374] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:41:09.711359 25654 raft_consensus.cc:729] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b187e4bc178e4af998b9cd688de15601, State: Initialized, Role: FOLLOWER
I20260810 06:41:09.711920 25654 consensus_queue.cc:260] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e4bc178e4af998b9cd688de15601" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 42153 } }
I20260810 06:41:09.712141 25654 raft_consensus.cc:388] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:41:09.712281 25654 raft_consensus.cc:482] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:41:09.712447 25654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:41:09.714749 25654 raft_consensus.cc:504] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e4bc178e4af998b9cd688de15601" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 42153 } }
I20260810 06:41:09.715201 25654 leader_election.cc:302] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b187e4bc178e4af998b9cd688de15601; no voters:
I20260810 06:41:09.715759 25654 leader_election.cc:288] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:41:09.715875 25660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:41:09.716499 25660 raft_consensus.cc:686] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 1 LEADER]: Becoming Leader. State: Replica: b187e4bc178e4af998b9cd688de15601, State: Running, Role: LEADER
I20260810 06:41:09.717016 25660 consensus_queue.cc:237] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e4bc178e4af998b9cd688de15601" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 42153 } }
I20260810 06:41:09.717639 25654 sys_catalog.cc:564] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:41:09.719658 25663 sys_catalog.cc:455] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b187e4bc178e4af998b9cd688de15601. Latest consensus state: current_term: 1 leader_uuid: "b187e4bc178e4af998b9cd688de15601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e4bc178e4af998b9cd688de15601" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 42153 } } }
I20260810 06:41:09.719724 25662 sys_catalog.cc:455] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b187e4bc178e4af998b9cd688de15601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e4bc178e4af998b9cd688de15601" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 42153 } } }
I20260810 06:41:09.719974 25663 sys_catalog.cc:458] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [sys.catalog]: This master's current role is: LEADER
I20260810 06:41:09.720057 25662 sys_catalog.cc:458] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [sys.catalog]: This master's current role is: LEADER
I20260810 06:41:09.728411 25671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:41:09.732832 25671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:41:09.741550 25671 catalog_manager.cc:1347] Generated new cluster ID: b7923ee04ef04a468f78e60a359e1f21
I20260810 06:41:09.741649 25671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:41:09.762077 25671 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:41:09.763341 25671 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:41:09.774346 25671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: Generated new TSK 0
I20260810 06:41:09.775132 25671 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
I20260810 06:41:09.838598 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:42153
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:41:10.170655 25696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:41:10.171064 25696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:41:10.171290 25696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:41:10.178556 25696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:41:10.178668 25696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:41:10.178788 25696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:41:10.188889 25696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:42153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:41:10.190135 25696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:41:10.191533 25696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:41:10.204236 25703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:10.204381 25704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:10.204870 25707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:41:10.205545 25696 server_base.cc:1034] running on GCE node
I20260810 06:41:10.206102 25696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:41:10.207017 25696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:41:10.208207 25696 hybrid_clock.cc:648] HybridClock initialized: now 1786344070208162 us; error 36 us; skew 500 ppm
I20260810 06:41:10.208513 25696 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:41:10 dist-test-slave-4tdj krb5kdc[25533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344070, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:41:10.211796 25696 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:41:10.213387 25696 webserver.cc:460] Webserver started at http://127.31.59.193:34765/ using document root <none> and password file <none>
I20260810 06:41:10.214130 25696 fs_manager.cc:359] Metadata directory not provided
I20260810 06:41:10.214258 25696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:41:10.214613 25696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:41:10.217720 25696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "5819bbcb73394a5e879808d977b00f96"
format_stamp: "Formatted at 2026-08-10 06:41:10 on dist-test-slave-4tdj"
server_key: "e6f733ce508ea1f6d4a2f0606bc1079f"
server_key_iv: "f39f2747aec5611437899b58f778c691"
server_key_version: "encryptionkey@0"
I20260810 06:41:10.218641 25696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "5819bbcb73394a5e879808d977b00f96"
format_stamp: "Formatted at 2026-08-10 06:41:10 on dist-test-slave-4tdj"
server_key: "e6f733ce508ea1f6d4a2f0606bc1079f"
server_key_iv: "f39f2747aec5611437899b58f778c691"
server_key_version: "encryptionkey@0"
I20260810 06:41:10.224009 25696 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 06:41:10.227528 25720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:10.228880 25696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:41:10.229048 25696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "5819bbcb73394a5e879808d977b00f96"
format_stamp: "Formatted at 2026-08-10 06:41:10 on dist-test-slave-4tdj"
server_key: "e6f733ce508ea1f6d4a2f0606bc1079f"
server_key_iv: "f39f2747aec5611437899b58f778c691"
server_key_version: "encryptionkey@0"
I20260810 06:41:10.229234 25696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:41:10.250516 25696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:41:10.254648 25696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:41:10.254937 25696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:41:10.255929 25696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:41:10.257325 25696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:41:10.257417 25696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:10.257565 25696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:41:10.257627 25696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:10.290508 25696 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:45199
I20260810 06:41:10.290586 25913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:45199 every 8 connection(s)
I20260810 06:41:10.291844 25696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:41:10.294167 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 25696
I20260810 06:41:10.294410 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:41:10.295230 31983 external_mini_cluster.cc:1428] Setting key ccdd19e47aa48bdcfe88da4a41eb2db5
Aug 10 06:41:10 dist-test-slave-4tdj krb5kdc[25533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344070, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:41:10.313778 25917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:10.294435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:41865 (local address 127.31.59.254:42153)
0810 06:41:10.294913 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:10.294918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:10.296097 (+ 1179us) server_negotiation.cc:407] Connection header received
0810 06:41:10.298416 (+ 2319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:10.298422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:10.298513 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:10.298623 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:10.300176 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:10.300913 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:10.302572 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:10.303065 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:10.303192 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:10.306987 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:10.306992 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:10.306995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:10.307037 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:10.310203 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:10.311021 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:10.311028 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:10.311032 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:10.311206 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:10.311693 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:10.311698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:10.311701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:10.312094 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:10.312246 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:10.313109 (+ 863us) server_negotiation.cc:299] Negotiation successful
0810 06:41:10.313346 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":240,"threads_started":1}
I20260810 06:41:10.314643 25918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:10.295060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:41865)
0810 06:41:10.295892 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:10.295952 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 06:41:10.298132 (+ 2180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:41:10.298878 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:41:10.298886 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:41:10.299410 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:41:10.299993 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:10.300012 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:10.301095 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:10.301099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:10.302414 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:10.302428 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:10.303323 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:10.303331 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:10.303588 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:10.304270 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:41:10.304302 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:41:10.306751 (+ 2449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:41:10.310388 (+ 3637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:41:10.310395 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:41:10.310406 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:41:10.310870 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:41:10.311315 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:41:10.311318 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:41:10.311321 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:41:10.311553 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:41:10.312248 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:41:10.312254 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:41:10.312646 (+ 392us) client_negotiation.cc:769] Sending connection context
0810 06:41:10.312922 (+ 276us) client_negotiation.cc:240] Negotiation successful
0810 06:41:10.313409 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":334,"threads_started":1}
I20260810 06:41:10.316320 25915 heartbeater.cc:347] Connected to a master server at 127.31.59.254:42153
I20260810 06:41:10.316694 25915 heartbeater.cc:464] Registering TS with master...
I20260810 06:41:10.317643 25915 heartbeater.cc:511] Master 127.31.59.254:42153 requested a full tablet report, sending...
I20260810 06:41:10.320142 25601 ts_manager.cc:194] Registered new tserver with Master: 5819bbcb73394a5e879808d977b00f96 (127.31.59.193:45199)
I20260810 06:41:10.321630 25601 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:41865
WARNING: no policy specified for kudu/127.31.59.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
I20260810 06:41:10.357424 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:42153
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:41:10.679622 25921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:41:10.679891 25921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:41:10.680032 25921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:41:10.687099 25921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:41:10.687216 25921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:41:10.687367 25921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:41:10.696772 25921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:42153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:41:10.697980 25921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:41:10.699313 25921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:41:10.712107 25931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:10.712184 25930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:10.712157 25935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:41:10.712949 25921 server_base.cc:1034] running on GCE node
I20260810 06:41:10.713609 25921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:41:10.714529 25921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:41:10.715716 25921 hybrid_clock.cc:648] HybridClock initialized: now 1786344070715667 us; error 45 us; skew 500 ppm
I20260810 06:41:10.715973 25921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:41:10 dist-test-slave-4tdj krb5kdc[25533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344070, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:41:10.719182 25921 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:41:10.720542 25921 webserver.cc:460] Webserver started at http://127.31.59.194:44731/ using document root <none> and password file <none>
I20260810 06:41:10.721195 25921 fs_manager.cc:359] Metadata directory not provided
I20260810 06:41:10.721309 25921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:41:10.721607 25921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:41:10.724403 25921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "387747fac59440b7986c72c220840f42"
format_stamp: "Formatted at 2026-08-10 06:41:10 on dist-test-slave-4tdj"
server_key: "f371baa3c1662cdbc4d8c8b3ec82decc"
server_key_iv: "f2af13bc6c7630f41a7cbe52b1db1bae"
server_key_version: "encryptionkey@0"
I20260810 06:41:10.725126 25921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "387747fac59440b7986c72c220840f42"
format_stamp: "Formatted at 2026-08-10 06:41:10 on dist-test-slave-4tdj"
server_key: "f371baa3c1662cdbc4d8c8b3ec82decc"
server_key_iv: "f2af13bc6c7630f41a7cbe52b1db1bae"
server_key_version: "encryptionkey@0"
I20260810 06:41:10.730274 25921 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 06:41:10.733619 25945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:10.734962 25921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 06:41:10.735097 25921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "387747fac59440b7986c72c220840f42"
format_stamp: "Formatted at 2026-08-10 06:41:10 on dist-test-slave-4tdj"
server_key: "f371baa3c1662cdbc4d8c8b3ec82decc"
server_key_iv: "f2af13bc6c7630f41a7cbe52b1db1bae"
server_key_version: "encryptionkey@0"
I20260810 06:41:10.735250 25921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:41:10.754405 25921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:41:10.758487 25921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:41:10.758757 25921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:41:10.759637 25921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:41:10.760908 25921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:41:10.760979 25921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:10.761092 25921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:41:10.761142 25921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:10.791306 25921 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:40509
I20260810 06:41:10.791352 26116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:40509 every 8 connection(s)
I20260810 06:41:10.792546 25921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:41:10.801335 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 25921
I20260810 06:41:10.801497 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:41:10.801937 31983 external_mini_cluster.cc:1428] Setting key d95b9089eb4c06f1eef2e299c6a8f4e6
Aug 10 06:41:10 dist-test-slave-4tdj krb5kdc[25533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344070, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:41:10.811986 25917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:10.794987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:57117 (local address 127.31.59.254:42153)
0810 06:41:10.795220 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:10.795227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:10.796328 (+ 1101us) server_negotiation.cc:407] Connection header received
0810 06:41:10.798070 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:10.798074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:10.798151 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:10.798238 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:10.799724 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:10.800439 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:10.801998 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:10.802638 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:10.802738 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:10.806140 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:10.806145 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:10.806150 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:10.806189 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:10.809014 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:10.809667 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:10.809672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:10.809675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:10.809830 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:10.810344 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:10.810349 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:10.810352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:10.810686 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:10.810830 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:10.811438 (+ 608us) server_negotiation.cc:299] Negotiation successful
0810 06:41:10.811681 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 06:41:10.813778 26121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:10.795455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:57117)
0810 06:41:10.796162 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:10.796206 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:41:10.797820 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:41:10.798466 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:41:10.798474 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:41:10.798998 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:41:10.799560 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:10.799587 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:10.800588 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:10.800592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:10.801858 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:10.801870 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:10.802859 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:10.802865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:10.803057 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:10.803714 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:41:10.803751 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:41:10.805909 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:41:10.809141 (+ 3232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:41:10.809147 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:41:10.809159 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:41:10.809555 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:41:10.810006 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:41:10.810010 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:41:10.810012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:41:10.810209 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:41:10.810798 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:41:10.810803 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:41:10.811105 (+ 302us) client_negotiation.cc:769] Sending connection context
0810 06:41:10.811329 (+ 224us) client_negotiation.cc:240] Negotiation successful
0810 06:41:10.811687 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":274,"threads_started":1}
I20260810 06:41:10.815371 26117 heartbeater.cc:347] Connected to a master server at 127.31.59.254:42153
I20260810 06:41:10.815675 26117 heartbeater.cc:464] Registering TS with master...
I20260810 06:41:10.816365 26117 heartbeater.cc:511] Master 127.31.59.254:42153 requested a full tablet report, sending...
I20260810 06:41:10.817832 25601 ts_manager.cc:194] Registered new tserver with Master: 387747fac59440b7986c72c220840f42 (127.31.59.194:40509)
I20260810 06:41:10.818686 25601 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:57117
WARNING: no policy specified for kudu/127.31.59.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
I20260810 06:41:10.859689 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:42153
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:41:11.241755 26125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:41:11.242110 26125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:41:11.242264 26125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:41:11.249449 26125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:41:11.249565 26125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:41:11.249713 26125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:41:11.259088 26125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:42153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:41:11.260281 26125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:41:11.261538 26125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:41:11.273689 26132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:11.273634 26139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:11.273599 26134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:41:11.274847 26125 server_base.cc:1034] running on GCE node
I20260810 06:41:11.275391 26125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:41:11.276247 26125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:41:11.277452 26125 hybrid_clock.cc:648] HybridClock initialized: now 1786344071277408 us; error 31 us; skew 500 ppm
I20260810 06:41:11.277743 26125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:41:11 dist-test-slave-4tdj krb5kdc[25533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344071, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:41:11.280735 26125 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:41:11.282099 26125 webserver.cc:460] Webserver started at http://127.31.59.195:34839/ using document root <none> and password file <none>
I20260810 06:41:11.282737 26125 fs_manager.cc:359] Metadata directory not provided
I20260810 06:41:11.282835 26125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:41:11.283092 26125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:41:11.285781 26125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "1d8cdef00ce34cc99e23974f6ec4a370"
format_stamp: "Formatted at 2026-08-10 06:41:11 on dist-test-slave-4tdj"
server_key: "9edda25b4dca6d30dbe0bea4583ba1a8"
server_key_iv: "6d24f9ed2a7a05617b33d90b3232fbc8"
server_key_version: "encryptionkey@0"
I20260810 06:41:11.286499 26125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "1d8cdef00ce34cc99e23974f6ec4a370"
format_stamp: "Formatted at 2026-08-10 06:41:11 on dist-test-slave-4tdj"
server_key: "9edda25b4dca6d30dbe0bea4583ba1a8"
server_key_iv: "6d24f9ed2a7a05617b33d90b3232fbc8"
server_key_version: "encryptionkey@0"
I20260810 06:41:11.291411 26125 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:41:11.294798 26151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:11.295997 26125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 06:41:11.296166 26125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "1d8cdef00ce34cc99e23974f6ec4a370"
format_stamp: "Formatted at 2026-08-10 06:41:11 on dist-test-slave-4tdj"
server_key: "9edda25b4dca6d30dbe0bea4583ba1a8"
server_key_iv: "6d24f9ed2a7a05617b33d90b3232fbc8"
server_key_version: "encryptionkey@0"
I20260810 06:41:11.296336 26125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:41:11.323840 26125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:41:11.325317 25915 heartbeater.cc:503] Master 127.31.59.254:42153 was elected leader, sending a full tablet report...
I20260810 06:41:11.328626 26125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:41:11.328863 26125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:41:11.329725 26125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:41:11.331032 26125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:41:11.331106 26125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:11.331193 26125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:41:11.331243 26125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:11.360059 26125 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:43191
I20260810 06:41:11.360096 26321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:43191 every 8 connection(s)
I20260810 06:41:11.361313 26125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:41:11.365180 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 26125
I20260810 06:41:11.365382 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:41:11.365805 31983 external_mini_cluster.cc:1428] Setting key b4f7887167e0471af1ca948e72118b82
Aug 10 06:41:11 dist-test-slave-4tdj krb5kdc[25533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344071, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:41:11.380628 26327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:11.363772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:34349 (local address 127.31.59.254:42153)
0810 06:41:11.364217 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:11.364222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:11.365168 (+ 946us) server_negotiation.cc:407] Connection header received
0810 06:41:11.367282 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:11.367287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:11.367367 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:11.367459 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:11.369184 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.369956 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.371552 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.371948 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.372056 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:11.375259 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:11.375263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:11.375266 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:11.375299 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:11.377711 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:11.378376 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:11.378380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:11.378382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:11.378501 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:11.378898 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:11.378901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:11.378903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:11.379155 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:11.379262 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:11.380058 (+ 796us) server_negotiation.cc:299] Negotiation successful
0810 06:41:11.380270 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":220,"threads_started":1}
I20260810 06:41:11.381470 26328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:11.364328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:34349)
0810 06:41:11.364994 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:11.365045 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 06:41:11.366997 (+ 1952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:41:11.367698 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:41:11.367705 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:41:11.368417 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:41:11.369013 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:11.369042 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.370110 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.370114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:11.371412 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:11.371425 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.372166 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.372175 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:11.372389 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:11.372945 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:41:11.372966 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:41:11.375071 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:41:11.377831 (+ 2760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:41:11.377838 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:41:11.377849 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:41:11.378251 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:41:11.378615 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:41:11.378619 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:41:11.378621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:41:11.378796 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:41:11.379275 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:41:11.379282 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:41:11.379630 (+ 348us) client_negotiation.cc:769] Sending connection context
0810 06:41:11.379909 (+ 279us) client_negotiation.cc:240] Negotiation successful
0810 06:41:11.380332 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":246,"threads_started":1}
I20260810 06:41:11.382750 26322 heartbeater.cc:347] Connected to a master server at 127.31.59.254:42153
I20260810 06:41:11.383035 26322 heartbeater.cc:464] Registering TS with master...
I20260810 06:41:11.383718 26322 heartbeater.cc:511] Master 127.31.59.254:42153 requested a full tablet report, sending...
I20260810 06:41:11.385037 25601 ts_manager.cc:194] Registered new tserver with Master: 1d8cdef00ce34cc99e23974f6ec4a370 (127.31.59.195:43191)
I20260810 06:41:11.385835 25601 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:34349
I20260810 06:41:11.393069 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:41:11.409415 26327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:11.397543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33868 (local address 127.31.59.254:42153)
0810 06:41:11.397732 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:11.397739 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:11.398172 (+ 433us) server_negotiation.cc:407] Connection header received
0810 06:41:11.398310 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:11.398316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:11.398405 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:11.398508 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:11.400057 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.400862 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.402141 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.402551 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.402640 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:11.404853 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:11.404858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:11.404861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:11.404892 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:11.407275 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:11.407862 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:11.407866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:11.407868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:11.407976 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:11.408279 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:11.408282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:11.408284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:11.408587 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:11.408749 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:11.409016 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 06:41:11.409172 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:41:11.413123 25601 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:41:11.420250 25601 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:41:11.456678 26342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:11.432597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:43191 (local address 127.0.0.1:40378)
0810 06:41:11.448434 (+ 15837us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:11.448452 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:41:11.448747 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:41:11.449614 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:41:11.449617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:41:11.449642 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:41:11.450045 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:11.450058 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.451813 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.451818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:11.453578 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:11.453591 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.455637 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.455644 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:11.455849 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:11.455899 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 06:41:11.456033 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 06:41:11.456232 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15713,"mutex_wait_us":37,"thread_start_us":15219,"threads_started":1}
I20260810 06:41:11.456669 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:11.432257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47486 (local address 127.31.59.194:40509)
0810 06:41:11.447053 (+ 14796us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:11.447063 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:11.448448 (+ 1385us) server_negotiation.cc:407] Connection header received
0810 06:41:11.448575 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:11.448584 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:11.448804 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:11.449025 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:41:11.449935 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.451158 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.453953 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.455329 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.455450 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:11.455634 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:11.456191 (+ 557us) server_negotiation.cc:299] Negotiation successful
0810 06:41:11.456349 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14618,"thread_start_us":14495,"threads_started":1}
I20260810 06:41:11.456686 26349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:11.433310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40378 (local address 127.31.59.195:43191)
0810 06:41:11.447971 (+ 14661us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:11.447980 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:11.448631 (+ 651us) server_negotiation.cc:407] Connection header received
0810 06:41:11.448965 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:11.448974 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:11.449182 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:11.449336 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:41:11.450237 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.451615 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.453728 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.455133 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.455258 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:11.455486 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:11.456109 (+ 623us) server_negotiation.cc:299] Negotiation successful
0810 06:41:11.456284 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14484,"thread_start_us":14315,"threads_started":1}
I20260810 06:41:11.456724 26343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:11.431768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48770 (local address 127.31.59.193:45199)
0810 06:41:11.447548 (+ 15780us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:11.447557 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:11.447590 (+ 33us) server_negotiation.cc:407] Connection header received
0810 06:41:11.447690 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:11.447699 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:11.447968 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:11.449083 (+ 1115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:41:11.449345 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.450834 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.454003 (+ 3169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.455175 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.455332 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:11.455569 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:11.456170 (+ 601us) server_negotiation.cc:299] Negotiation successful
0810 06:41:11.456403 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15594,"thread_start_us":15448,"threads_started":1}
I20260810 06:41:11.457315 26345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:11.431673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:40509 (local address 127.0.0.1:47486)
0810 06:41:11.447945 (+ 16272us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:11.447981 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:41:11.448305 (+ 324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:41:11.449274 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:41:11.449278 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:41:11.449303 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:41:11.449721 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:11.449738 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.451341 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.451347 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:11.453788 (+ 2441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:11.453798 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.455781 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.455788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:11.455941 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:11.455967 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 06:41:11.456033 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 06:41:11.456975 (+ 942us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16146,"mutex_wait_us":794,"thread_start_us":16465,"threads_started":1}
I20260810 06:41:11.457651 26340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:11.431589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45199 (local address 127.0.0.1:48770)
0810 06:41:11.446739 (+ 15150us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:11.446799 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 06:41:11.447008 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:41:11.448450 (+ 1442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:41:11.448457 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:41:11.448489 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:41:11.448883 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:11.448896 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.450979 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.450985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:11.453781 (+ 2796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:11.453797 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.455742 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.455749 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:11.455935 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:11.455971 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:41:11.456066 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 06:41:11.457366 (+ 1300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14838,"mutex_wait_us":1272,"thread_start_us":15424,"threads_started":1}
I20260810 06:41:11.461407 25805 tablet_service.cc:1467] Processing CreateTablet for tablet 32b447a1a1f6479b91817c135ef9a7fe (DEFAULT_TABLE table=test-table [id=04e60ea1c09245fa8f88b8a9ac650d3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:41:11.461469 26019 tablet_service.cc:1467] Processing CreateTablet for tablet 32b447a1a1f6479b91817c135ef9a7fe (DEFAULT_TABLE table=test-table [id=04e60ea1c09245fa8f88b8a9ac650d3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:41:11.461469 26224 tablet_service.cc:1467] Processing CreateTablet for tablet 32b447a1a1f6479b91817c135ef9a7fe (DEFAULT_TABLE table=test-table [id=04e60ea1c09245fa8f88b8a9ac650d3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:41:11.462756 25805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32b447a1a1f6479b91817c135ef9a7fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:41:11.462836 26019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32b447a1a1f6479b91817c135ef9a7fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:41:11.462836 26224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32b447a1a1f6479b91817c135ef9a7fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:41:11.474455 26353 tablet_bootstrap.cc:492] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96: Bootstrap starting.
I20260810 06:41:11.476701 26357 tablet_bootstrap.cc:492] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370: Bootstrap starting.
I20260810 06:41:11.477506 26358 tablet_bootstrap.cc:492] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42: Bootstrap starting.
I20260810 06:41:11.478529 26353 tablet_bootstrap.cc:654] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96: Neither blocks nor log segments found. Creating new log.
I20260810 06:41:11.479938 26353 log.cc:824] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96: Log is configured to *not* fsync() on all Append() calls
I20260810 06:41:11.480841 26357 tablet_bootstrap.cc:654] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370: Neither blocks nor log segments found. Creating new log.
I20260810 06:41:11.481247 26358 tablet_bootstrap.cc:654] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42: Neither blocks nor log segments found. Creating new log.
I20260810 06:41:11.481922 26357 log.cc:824] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370: Log is configured to *not* fsync() on all Append() calls
I20260810 06:41:11.482534 26358 log.cc:824] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42: Log is configured to *not* fsync() on all Append() calls
I20260810 06:41:11.483395 26353 tablet_bootstrap.cc:492] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96: No bootstrap required, opened a new log
I20260810 06:41:11.483758 26353 ts_tablet_manager.cc:1397] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260810 06:41:11.485424 26358 tablet_bootstrap.cc:492] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42: No bootstrap required, opened a new log
I20260810 06:41:11.485718 26358 ts_tablet_manager.cc:1397] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260810 06:41:11.485822 26357 tablet_bootstrap.cc:492] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370: No bootstrap required, opened a new log
I20260810 06:41:11.486055 26357 ts_tablet_manager.cc:1397] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260810 06:41:11.491420 26353 raft_consensus.cc:348] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } }
I20260810 06:41:11.491798 26353 raft_consensus.cc:374] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:41:11.491909 26353 raft_consensus.cc:729] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5819bbcb73394a5e879808d977b00f96, State: Initialized, Role: FOLLOWER
I20260810 06:41:11.492506 26353 consensus_queue.cc:260] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } }
I20260810 06:41:11.495489 26353 ts_tablet_manager.cc:1428] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96: Time spent starting tablet: real 0.012s user 0.009s sys 0.002s
I20260810 06:41:11.496251 26357 raft_consensus.cc:348] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } }
I20260810 06:41:11.496251 26358 raft_consensus.cc:348] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } }
I20260810 06:41:11.496676 26358 raft_consensus.cc:374] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:41:11.496739 26357 raft_consensus.cc:374] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:41:11.496856 26357 raft_consensus.cc:729] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d8cdef00ce34cc99e23974f6ec4a370, State: Initialized, Role: FOLLOWER
I20260810 06:41:11.497519 26358 raft_consensus.cc:729] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 387747fac59440b7986c72c220840f42, State: Initialized, Role: FOLLOWER
I20260810 06:41:11.497468 26357 consensus_queue.cc:260] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } }
I20260810 06:41:11.498250 26358 consensus_queue.cc:260] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } }
I20260810 06:41:11.498678 26322 heartbeater.cc:503] Master 127.31.59.254:42153 was elected leader, sending a full tablet report...
I20260810 06:41:11.499236 26357 ts_tablet_manager.cc:1428] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260810 06:41:11.499341 26117 heartbeater.cc:503] Master 127.31.59.254:42153 was elected leader, sending a full tablet report...
I20260810 06:41:11.501015 26358 ts_tablet_manager.cc:1428] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
W20260810 06:41:11.544140 25916 tablet.cc:2369] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:41:11.544334 26119 tablet.cc:2369] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:41:11.612886 26324 tablet.cc:2369] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:41:11.636375 26368 raft_consensus.cc:482] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:41:11.636698 26368 raft_consensus.cc:504] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } }
I20260810 06:41:11.638490 26368 leader_election.cc:288] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5819bbcb73394a5e879808d977b00f96 (127.31.59.193:45199), 387747fac59440b7986c72c220840f42 (127.31.59.194:40509)
I20260810 06:41:11.644758 26328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:11.638766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:40509 (local address 127.31.59.195:45825)
0810 06:41:11.639062 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:11.639098 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:41:11.639212 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:41:11.639609 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:41:11.639614 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:41:11.639649 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:41:11.639848 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:11.639860 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.641133 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.641139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:11.642651 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:11.642664 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.643999 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.644006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:11.644194 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:11.644229 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 06:41:11.644343 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 06:41:11.644475 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260810 06:41:11.644757 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:11.639043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:45825 (local address 127.31.59.194:40509)
0810 06:41:11.639240 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:11.639246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:11.639268 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:41:11.639338 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:11.639344 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:11.639443 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:11.639529 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:41:11.640009 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.641007 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.642801 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.643851 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.643959 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:11.644011 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:11.644402 (+ 391us) server_negotiation.cc:299] Negotiation successful
0810 06:41:11.644531 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:41:11.645538 26343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:11.639029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:52377 (local address 127.31.59.193:45199)
0810 06:41:11.639244 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:11.639251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:11.639883 (+ 632us) server_negotiation.cc:407] Connection header received
0810 06:41:11.640006 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:11.640012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:11.640082 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:11.640146 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:41:11.640702 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.641629 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.643564 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.644615 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.644761 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:11.644830 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:11.645252 (+ 422us) server_negotiation.cc:299] Negotiation successful
0810 06:41:11.645345 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 06:41:11.645676 26370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:11.638833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45199 (local address 127.31.59.195:52377)
0810 06:41:11.639790 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:11.639806 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:41:11.639899 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:41:11.640184 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:41:11.640188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:41:11.640213 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:41:11.640552 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:11.640563 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.641778 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.641782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:11.643412 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:11.643426 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.644849 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.644857 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:11.645055 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:11.645095 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 06:41:11.645207 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 06:41:11.645348 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"thread_start_us":598,"threads_started":1}
I20260810 06:41:11.646008 26048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32b447a1a1f6479b91817c135ef9a7fe" candidate_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "387747fac59440b7986c72c220840f42" is_pre_election: true
I20260810 06:41:11.646577 26048 raft_consensus.cc:2454] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d8cdef00ce34cc99e23974f6ec4a370 in term 0.
I20260810 06:41:11.646780 25836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32b447a1a1f6479b91817c135ef9a7fe" candidate_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5819bbcb73394a5e879808d977b00f96" is_pre_election: true
I20260810 06:41:11.647308 26155 leader_election.cc:302] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d8cdef00ce34cc99e23974f6ec4a370, 387747fac59440b7986c72c220840f42; no voters:
I20260810 06:41:11.647324 25836 raft_consensus.cc:2454] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d8cdef00ce34cc99e23974f6ec4a370 in term 0.
I20260810 06:41:11.647745 26368 raft_consensus.cc:2781] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:41:11.647928 26368 raft_consensus.cc:482] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:41:11.648057 26368 raft_consensus.cc:3037] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:41:11.650447 26368 raft_consensus.cc:504] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } }
I20260810 06:41:11.651098 26368 leader_election.cc:288] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [CANDIDATE]: Term 1 election: Requested vote from peers 5819bbcb73394a5e879808d977b00f96 (127.31.59.193:45199), 387747fac59440b7986c72c220840f42 (127.31.59.194:40509)
I20260810 06:41:11.651612 26048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32b447a1a1f6479b91817c135ef9a7fe" candidate_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "387747fac59440b7986c72c220840f42"
I20260810 06:41:11.651612 25836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32b447a1a1f6479b91817c135ef9a7fe" candidate_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5819bbcb73394a5e879808d977b00f96"
I20260810 06:41:11.651837 25836 raft_consensus.cc:3037] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:41:11.651837 26048 raft_consensus.cc:3037] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:41:11.655141 26048 raft_consensus.cc:2454] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d8cdef00ce34cc99e23974f6ec4a370 in term 1.
I20260810 06:41:11.655141 25836 raft_consensus.cc:2454] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d8cdef00ce34cc99e23974f6ec4a370 in term 1.
I20260810 06:41:11.655582 26156 leader_election.cc:302] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d8cdef00ce34cc99e23974f6ec4a370, 5819bbcb73394a5e879808d977b00f96; no voters:
I20260810 06:41:11.655884 26368 raft_consensus.cc:2781] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:41:11.656308 26368 raft_consensus.cc:686] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [term 1 LEADER]: Becoming Leader. State: Replica: 1d8cdef00ce34cc99e23974f6ec4a370, State: Running, Role: LEADER
I20260810 06:41:11.656761 26368 consensus_queue.cc:237] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } }
I20260810 06:41:11.661197 25601 catalog_manager.cc:5495] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d8cdef00ce34cc99e23974f6ec4a370 (127.31.59.195). New cstate: current_term: 1 leader_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } health_report { overall_health: HEALTHY } } }
I20260810 06:41:11.693535 26349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:11.688219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40386 (local address 127.31.59.195:43191)
0810 06:41:11.688400 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:11.688405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:11.688423 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:41:11.688570 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:11.688573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:11.688647 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:11.688766 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:41:11.689100 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.689821 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.691489 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.691895 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.691986 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:11.692486 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:41:11.692987 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:41:11.693096 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:11.693239 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 06:41:11.693301 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:41:11.701607 26048 raft_consensus.cc:1261] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42 [term 1 FOLLOWER]: Refusing update from remote peer 1d8cdef00ce34cc99e23974f6ec4a370: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:41:11.701627 25836 raft_consensus.cc:1261] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Refusing update from remote peer 1d8cdef00ce34cc99e23974f6ec4a370: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:41:11.702530 26368 consensus_queue.cc:1035] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:41:11.702807 26371 consensus_queue.cc:1035] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [LEADER]: Connected to new peer: Peer: permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:41:11.715039 26374 mvcc.cc:205] Tried to move back new op lower bound from 7316865317679792128 to 7316865317509804032. Current Snapshot: MvccSnapshot[applied={T|T < 7316865317679792128}]
I20260810 06:41:11.726630 25601 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:41:11.726868 25601 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:41:11.730379 25601 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: Sending DeleteTablet for 3 replicas of tablet 32b447a1a1f6479b91817c135ef9a7fe
I20260810 06:41:11.731473 25805 tablet_service.cc:1514] Processing DeleteTablet for tablet 32b447a1a1f6479b91817c135ef9a7fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:41:11 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:48770
I20260810 06:41:11.731603 26019 tablet_service.cc:1514] Processing DeleteTablet for tablet 32b447a1a1f6479b91817c135ef9a7fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:41:11 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:47486
I20260810 06:41:11.731796 26224 tablet_service.cc:1514] Processing DeleteTablet for tablet 32b447a1a1f6479b91817c135ef9a7fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:41:11 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:40378
I20260810 06:41:11.732933 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 25696
I20260810 06:41:11.747054 26392 tablet_replica.cc:330] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42: stopping tablet replica
I20260810 06:41:11.747551 26392 raft_consensus.cc:2229] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:41:11.748092 26392 raft_consensus.cc:2258] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:41:11.752108 26392 ts_tablet_manager.cc:1905] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:41:11.759397 26392 ts_tablet_manager.cc:1918] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:41:11.759637 26392 log.cc:1196] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/32b447a1a1f6479b91817c135ef9a7fe
I20260810 06:41:11.760196 26393 tablet_replica.cc:330] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370: stopping tablet replica
I20260810 06:41:11.760281 26392 ts_tablet_manager.cc:1939] T 32b447a1a1f6479b91817c135ef9a7fe P 387747fac59440b7986c72c220840f42: Deleting consensus metadata
I20260810 06:41:11.762152 25581 catalog_manager.cc:4841] TS 387747fac59440b7986c72c220840f42 (127.31.59.194:40509): tablet 32b447a1a1f6479b91817c135ef9a7fe (table test-table [id=04e60ea1c09245fa8f88b8a9ac650d3e]) successfully deleted
I20260810 06:41:11.762737 26393 raft_consensus.cc:2229] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:41:11.763142 26391 tablet_replica.cc:330] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96: stopping tablet replica
I20260810 06:41:11.763506 26393 raft_consensus.cc:2258] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:41:11.763603 26391 raft_consensus.cc:2229] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:41:11.764400 26391 raft_consensus.cc:2258] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:41:11.765709 26393 ts_tablet_manager.cc:1905] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:41:11.766819 26391 ts_tablet_manager.cc:1905] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 06:41:11 dist-test-slave-4tdj krb5kdc[25533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:41:11.773691 26391 ts_tablet_manager.cc:1918] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:41:11.773768 26393 ts_tablet_manager.cc:1918] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:41:11.773931 26393 log.cc:1196] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/32b447a1a1f6479b91817c135ef9a7fe
I20260810 06:41:11.773994 26391 log.cc:1196] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/32b447a1a1f6479b91817c135ef9a7fe
I20260810 06:41:11.774621 26393 ts_tablet_manager.cc:1939] T 32b447a1a1f6479b91817c135ef9a7fe P 1d8cdef00ce34cc99e23974f6ec4a370: Deleting consensus metadata
I20260810 06:41:11.774639 26391 ts_tablet_manager.cc:1939] T 32b447a1a1f6479b91817c135ef9a7fe P 5819bbcb73394a5e879808d977b00f96: Deleting consensus metadata
I20260810 06:41:11.776403 25579 catalog_manager.cc:4841] TS 1d8cdef00ce34cc99e23974f6ec4a370 (127.31.59.195:43191): tablet 32b447a1a1f6479b91817c135ef9a7fe (table test-table [id=04e60ea1c09245fa8f88b8a9ac650d3e]) successfully deleted
I20260810 06:41:11.776383 25582 catalog_manager.cc:4841] TS 5819bbcb73394a5e879808d977b00f96 (127.31.59.193:45199): tablet 32b447a1a1f6479b91817c135ef9a7fe (table test-table [id=04e60ea1c09245fa8f88b8a9ac650d3e]) successfully deleted
I20260810 06:41:11.778215 26343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:11.734247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48772 (local address 127.31.59.193:45199)
0810 06:41:11.734453 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:11.734459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:11.756942 (+ 22483us) server_negotiation.cc:407] Connection header received
0810 06:41:11.757429 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:11.757436 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:11.757520 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:11.757613 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:11.760420 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.761538 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.764309 (+ 2771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.765045 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.765159 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:11.770815 (+ 5656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:11.770832 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:11.770850 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:11.770901 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:11.775053 (+ 4152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:11.776113 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:11.776124 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:11.776128 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:11.776306 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:11.776833 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:11.776840 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:11.776842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:11.777206 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:11.777422 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:11.777778 (+ 356us) server_negotiation.cc:299] Negotiation successful
0810 06:41:11.777933 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 06:41:11.778934 25909 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:41:11.903697 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 25696
I20260810 06:41:11.927909 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 25921
Aug 10 06:41:11 dist-test-slave-4tdj krb5kdc[25533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.194@KRBTEST.COM
I20260810 06:41:11.944706 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:11.928798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47498 (local address 127.31.59.194:40509)
0810 06:41:11.929012 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:11.929019 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:11.929042 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:41:11.929203 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:11.929208 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:11.929281 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:11.929377 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:11.930943 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.931737 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.933294 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:11.933714 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:11.933836 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:11.938769 (+ 4933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:11.938788 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:11.938808 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:11.938850 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:11.941909 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:11.942629 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:11.942636 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:11.942639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:11.942763 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:11.943162 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:11.943166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:11.943168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:11.943574 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:11.943861 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:11.944163 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 06:41:11.944375 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 06:41:11.945420 26111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:41:12.033960 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 25921
I20260810 06:41:12.056957 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 26125
Aug 10 06:41:12 dist-test-slave-4tdj krb5kdc[25533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.195@KRBTEST.COM
I20260810 06:41:12.073025 26349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:12.057729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40396 (local address 127.31.59.195:43191)
0810 06:41:12.057924 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:12.057929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:12.058073 (+ 144us) server_negotiation.cc:407] Connection header received
0810 06:41:12.058276 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:12.058280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:12.058401 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:12.058520 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:12.059794 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:12.060599 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:12.061811 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:12.062257 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:12.062422 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:12.066883 (+ 4461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:12.066899 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:12.066915 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:12.066953 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:12.069984 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:12.070914 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:12.070921 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:12.070924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:12.071092 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:12.071504 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:12.071529 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:12.071531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:12.071860 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:12.072086 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:12.072496 (+ 410us) server_negotiation.cc:299] Negotiation successful
0810 06:41:12.072675 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 06:41:12.073807 26317 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:41:12.196583 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 26125
I20260810 06:41:12.224632 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 25557
I20260810 06:41:12.225958 25646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:41:12.334267 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 25557
I20260810 06:41:32.361896 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:42153
--webserver_interface=127.31.59.254
--webserver_port=43485
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:42153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:41:32.694651 26404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:41:32.694917 26404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:41:32.695041 26404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:41:32.701918 26404 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 06:41:32.701995 26404 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 06:41:32.702044 26404 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 06:41:32.702112 26404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:41:32.702179 26404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:41:32.702270 26404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:41:32.702338 26404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:41:32.702411 26404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:41:32.711761 26404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:42153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:42153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=43485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:41:32.712946 26404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:41:32.714237 26404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:41:32.737727 26417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:32.737672 26423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:32.737694 26420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:41:32.738700 26404 server_base.cc:1034] running on GCE node
I20260810 06:41:32.739559 26404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:41:32.741092 26404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:41:32.742357 26404 hybrid_clock.cc:648] HybridClock initialized: now 1786344092742251 us; error 46 us; skew 500 ppm
I20260810 06:41:32.742749 26404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:41:32 dist-test-slave-4tdj krb5kdc[25533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344092, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:41:32.746167 26404 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:41:32.747753 26404 webserver.cc:460] Webserver started at http://127.31.59.254:43485/ using document root <none> and password file <none>
I20260810 06:41:32.748510 26404 fs_manager.cc:359] Metadata directory not provided
I20260810 06:41:32.748633 26404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:41:32.754403 26404 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.002s
I20260810 06:41:32.757382 26432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:32.758793 26404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260810 06:41:32.758981 26404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "b187e4bc178e4af998b9cd688de15601"
format_stamp: "Formatted at 2026-08-10 06:41:09 on dist-test-slave-4tdj"
server_key: "1393d5f8b33cd7161f07558038fcbbd1"
server_key_iv: "bebe20a558133f48e3451e1e52e81900"
server_key_version: "encryptionkey@0"
I20260810 06:41:32.760033 26404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:41:32.773842 26404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:41:32.778030 26404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:41:32.778339 26404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:41:32.795784 26404 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:42153
I20260810 06:41:32.795821 26530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:42153 every 8 connection(s)
I20260810 06:41:32.797350 26404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:41:32.801263 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 26404
I20260810 06:41:32.802254 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:45199
--local_ip_for_outbound_sockets=127.31.59.193
--tserver_master_addrs=127.31.59.254:42153
--webserver_port=34765
--webserver_interface=127.31.59.193
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:41:32.805498 26533 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:41:32.809756 26533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: Bootstrap starting.
I20260810 06:41:32.830578 26533 log.cc:824] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: Log is configured to *not* fsync() on all Append() calls
I20260810 06:41:32.843137 26533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:41:32.843634 26533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: Bootstrap complete.
I20260810 06:41:32.852684 26533 raft_consensus.cc:348] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e4bc178e4af998b9cd688de15601" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 42153 } }
I20260810 06:41:32.853530 26533 raft_consensus.cc:729] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b187e4bc178e4af998b9cd688de15601, State: Initialized, Role: FOLLOWER
I20260810 06:41:32.854166 26533 consensus_queue.cc:260] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e4bc178e4af998b9cd688de15601" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 42153 } }
I20260810 06:41:32.854462 26533 raft_consensus.cc:388] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:41:32.854578 26533 raft_consensus.cc:482] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:41:32.854743 26533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:41:32.858482 26533 raft_consensus.cc:504] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e4bc178e4af998b9cd688de15601" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 42153 } }
I20260810 06:41:32.858986 26533 leader_election.cc:302] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b187e4bc178e4af998b9cd688de15601; no voters:
I20260810 06:41:32.859607 26533 leader_election.cc:288] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:41:32.859747 26541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:41:32.860857 26541 raft_consensus.cc:686] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 2 LEADER]: Becoming Leader. State: Replica: b187e4bc178e4af998b9cd688de15601, State: Running, Role: LEADER
I20260810 06:41:32.861460 26541 consensus_queue.cc:237] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e4bc178e4af998b9cd688de15601" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 42153 } }
I20260810 06:41:32.861832 26533 sys_catalog.cc:564] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:41:32.864094 26543 sys_catalog.cc:455] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b187e4bc178e4af998b9cd688de15601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e4bc178e4af998b9cd688de15601" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 42153 } } }
I20260810 06:41:32.864459 26543 sys_catalog.cc:458] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [sys.catalog]: This master's current role is: LEADER
I20260810 06:41:32.865041 26545 sys_catalog.cc:455] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b187e4bc178e4af998b9cd688de15601. Latest consensus state: current_term: 2 leader_uuid: "b187e4bc178e4af998b9cd688de15601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e4bc178e4af998b9cd688de15601" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 42153 } } }
I20260810 06:41:32.865346 26545 sys_catalog.cc:458] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [sys.catalog]: This master's current role is: LEADER
I20260810 06:41:32.873924 26551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:41:32.881547 26551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:41:32.882951 26551 catalog_manager.cc:1259] Loaded cluster ID: b7923ee04ef04a468f78e60a359e1f21
I20260810 06:41:32.883104 26551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:41:32.887073 26551 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:41:32.888559 26551 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: Loaded TSK: 0
I20260810 06:41:32.926973 26551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: Generated new TSK 1
I20260810 06:41:32.927387 26551 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:41:33.158851 26537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:41:33.159166 26537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:41:33.159296 26537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:41:33.166527 26537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:41:33.166631 26537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:41:33.166761 26537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:41:33.176389 26537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:45199
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=34765
--tserver_master_addrs=127.31.59.254:42153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:41:33.177644 26537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:41:33.178987 26537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:41:33.193046 26585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:33.193252 26586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:33.194078 26588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:41:33.194432 26537 server_base.cc:1034] running on GCE node
I20260810 06:41:33.195219 26537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:41:33.196401 26537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:41:33.197636 26537 hybrid_clock.cc:648] HybridClock initialized: now 1786344093197587 us; error 34 us; skew 500 ppm
I20260810 06:41:33.197999 26537 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:41:33 dist-test-slave-4tdj krb5kdc[25533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344093, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:41:33.201655 26537 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:41:33.203234 26537 webserver.cc:460] Webserver started at http://127.31.59.193:34765/ using document root <none> and password file <none>
I20260810 06:41:33.204020 26537 fs_manager.cc:359] Metadata directory not provided
I20260810 06:41:33.204172 26537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:41:33.210870 26537 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260810 06:41:33.214728 26604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:33.216125 26537 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 06:41:33.216288 26537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "5819bbcb73394a5e879808d977b00f96"
format_stamp: "Formatted at 2026-08-10 06:41:10 on dist-test-slave-4tdj"
server_key: "e6f733ce508ea1f6d4a2f0606bc1079f"
server_key_iv: "f39f2747aec5611437899b58f778c691"
server_key_version: "encryptionkey@0"
I20260810 06:41:33.217244 26537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:41:33.246148 26537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:41:33.249914 26537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:41:33.250185 26537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:41:33.251207 26537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:41:33.252486 26537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:41:33.252583 26537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:33.252689 26537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:41:33.252760 26537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:33.282454 26537 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:45199
I20260810 06:41:33.282537 26790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:45199 every 8 connection(s)
I20260810 06:41:33.283794 26537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:41:33.291397 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 26537
I20260810 06:41:33.292004 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:40509
--local_ip_for_outbound_sockets=127.31.59.194
--tserver_master_addrs=127.31.59.254:42153
--webserver_port=44731
--webserver_interface=127.31.59.194
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:41:33 dist-test-slave-4tdj krb5kdc[25533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344093, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:41:33.310770 26795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:33.286427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:42243 (local address 127.31.59.254:42153)
0810 06:41:33.286969 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:33.286980 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:33.287580 (+ 600us) server_negotiation.cc:407] Connection header received
0810 06:41:33.289611 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:33.289623 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:33.289918 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:33.290201 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:33.291962 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:33.293809 (+ 1847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:33.296206 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:33.297244 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:33.297442 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:33.302063 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:33.302072 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:33.302090 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:33.302139 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:33.306146 (+ 4007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:33.306959 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:33.306968 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:33.306976 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:33.307188 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:33.307716 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:33.307721 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:33.307724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:33.308334 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:33.308522 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:33.309480 (+ 958us) server_negotiation.cc:299] Negotiation successful
0810 06:41:33.309791 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":232,"threads_started":1}
I20260810 06:41:33.311008 26796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:33.286644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:42243)
0810 06:41:33.287422 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:33.287465 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:41:33.289155 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:41:33.290367 (+ 1212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:41:33.290377 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:41:33.291047 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:41:33.291736 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:33.291758 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:33.294046 (+ 2288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:33.294052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:33.295997 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:33.296024 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:33.297815 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:33.297823 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:33.298132 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:33.298870 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:41:33.298900 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:41:33.301759 (+ 2859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:41:33.306278 (+ 4519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:41:33.306287 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:41:33.306333 (+ 46us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:41:33.306816 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:41:33.307324 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:41:33.307329 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:41:33.307332 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:41:33.307575 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:41:33.308600 (+ 1025us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:41:33.308607 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:41:33.308974 (+ 367us) client_negotiation.cc:769] Sending connection context
0810 06:41:33.309293 (+ 319us) client_negotiation.cc:240] Negotiation successful
0810 06:41:33.309797 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":307,"threads_started":1}
I20260810 06:41:33.313987 26792 heartbeater.cc:347] Connected to a master server at 127.31.59.254:42153
I20260810 06:41:33.314276 26792 heartbeater.cc:464] Registering TS with master...
I20260810 06:41:33.315018 26792 heartbeater.cc:511] Master 127.31.59.254:42153 requested a full tablet report, sending...
I20260810 06:41:33.317355 26466 ts_manager.cc:194] Registered new tserver with Master: 5819bbcb73394a5e879808d977b00f96 (127.31.59.193:45199)
I20260810 06:41:33.319557 26466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:42243
W20260810 06:41:33.621474 26797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:41:33.621760 26797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:41:33.621889 26797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:41:33.628823 26797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:41:33.628918 26797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:41:33.629033 26797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:41:33.638182 26797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:40509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=44731
--tserver_master_addrs=127.31.59.254:42153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:41:33.639370 26797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:41:33.640581 26797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:41:33.652956 26804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:33.652920 26809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:33.652879 26806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:41:33.653894 26797 server_base.cc:1034] running on GCE node
I20260810 06:41:33.654551 26797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:41:33.655438 26797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:41:33.656623 26797 hybrid_clock.cc:648] HybridClock initialized: now 1786344093656565 us; error 51 us; skew 500 ppm
I20260810 06:41:33.656917 26797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:41:33 dist-test-slave-4tdj krb5kdc[25533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344093, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:41:33.659902 26797 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:41:33.661197 26797 webserver.cc:460] Webserver started at http://127.31.59.194:44731/ using document root <none> and password file <none>
I20260810 06:41:33.661815 26797 fs_manager.cc:359] Metadata directory not provided
I20260810 06:41:33.661947 26797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:41:33.667409 26797 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:41:33.669965 26819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:33.671262 26797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:41:33.671411 26797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "387747fac59440b7986c72c220840f42"
format_stamp: "Formatted at 2026-08-10 06:41:10 on dist-test-slave-4tdj"
server_key: "f371baa3c1662cdbc4d8c8b3ec82decc"
server_key_iv: "f2af13bc6c7630f41a7cbe52b1db1bae"
server_key_version: "encryptionkey@0"
I20260810 06:41:33.672358 26797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:41:33.684471 26797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:41:33.687983 26797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:41:33.688216 26797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:41:33.689106 26797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:41:33.690371 26797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:41:33.690469 26797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:33.690568 26797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:41:33.690636 26797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:33.720635 26797 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:40509
I20260810 06:41:33.720726 27005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:40509 every 8 connection(s)
I20260810 06:41:33.722111 26797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:41:33.723698 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 26797
I20260810 06:41:33.724306 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:43191
--local_ip_for_outbound_sockets=127.31.59.195
--tserver_master_addrs=127.31.59.254:42153
--webserver_port=34839
--webserver_interface=127.31.59.195
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:41:33 dist-test-slave-4tdj krb5kdc[25533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344093, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:41:33.743258 26795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:33.724804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:44723 (local address 127.31.59.254:42153)
0810 06:41:33.725014 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:33.725020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:33.726245 (+ 1225us) server_negotiation.cc:407] Connection header received
0810 06:41:33.728546 (+ 2301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:33.728550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:33.728625 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:33.728714 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:33.730702 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:33.731403 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:33.733484 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:33.733918 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:33.734023 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:33.737745 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:33.737750 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:33.737753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:33.737789 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:33.740234 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:33.740963 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:33.740968 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:33.740971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:33.741106 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:33.741529 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:33.741533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:33.741535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:33.741808 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:33.741943 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:33.742680 (+ 737us) server_negotiation.cc:299] Negotiation successful
0810 06:41:33.742920 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:41:33.744207 27010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:33.725238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:44723)
0810 06:41:33.726057 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:33.726109 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 06:41:33.728243 (+ 2134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:41:33.728966 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:41:33.728976 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:41:33.729677 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:41:33.730485 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:33.730510 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:33.731565 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:33.731570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:33.733311 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:33.733325 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:33.734137 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:33.734144 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:33.734452 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:33.735147 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:41:33.735175 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:41:33.737514 (+ 2339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:41:33.740382 (+ 2868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:41:33.740390 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:41:33.740407 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:41:33.740835 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:41:33.741211 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:41:33.741216 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:41:33.741220 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:41:33.741408 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:41:33.741937 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:41:33.741944 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:41:33.742268 (+ 324us) client_negotiation.cc:769] Sending connection context
0810 06:41:33.742560 (+ 292us) client_negotiation.cc:240] Negotiation successful
0810 06:41:33.743107 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":298,"threads_started":1}
I20260810 06:41:33.745635 27006 heartbeater.cc:347] Connected to a master server at 127.31.59.254:42153
I20260810 06:41:33.745937 27006 heartbeater.cc:464] Registering TS with master...
I20260810 06:41:33.746714 27006 heartbeater.cc:511] Master 127.31.59.254:42153 requested a full tablet report, sending...
I20260810 06:41:33.748143 26466 ts_manager.cc:194] Registered new tserver with Master: 387747fac59440b7986c72c220840f42 (127.31.59.194:40509)
I20260810 06:41:33.748960 26466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:44723
W20260810 06:41:34.054999 27011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:41:34.055258 27011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:41:34.055388 27011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:41:34.062704 27011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:41:34.062831 27011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:41:34.063022 27011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:41:34.072638 27011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:43191
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=34839
--tserver_master_addrs=127.31.59.254:42153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:41:34.073807 27011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:41:34.075153 27011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:41:34.087957 27023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:34.088011 27020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:34.088034 27021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:41:34.088956 27011 server_base.cc:1034] running on GCE node
I20260810 06:41:34.089489 27011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:41:34.090377 27011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:41:34.091634 27011 hybrid_clock.cc:648] HybridClock initialized: now 1786344094091575 us; error 44 us; skew 500 ppm
I20260810 06:41:34.091943 27011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:41:34 dist-test-slave-4tdj krb5kdc[25533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344094, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:41:34.095047 27011 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:41:34.096469 27011 webserver.cc:460] Webserver started at http://127.31.59.195:34839/ using document root <none> and password file <none>
I20260810 06:41:34.097124 27011 fs_manager.cc:359] Metadata directory not provided
I20260810 06:41:34.097255 27011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:41:34.102262 27011 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:41:34.104812 27037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:34.106041 27011 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:41:34.106185 27011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "1d8cdef00ce34cc99e23974f6ec4a370"
format_stamp: "Formatted at 2026-08-10 06:41:11 on dist-test-slave-4tdj"
server_key: "9edda25b4dca6d30dbe0bea4583ba1a8"
server_key_iv: "6d24f9ed2a7a05617b33d90b3232fbc8"
server_key_version: "encryptionkey@0"
I20260810 06:41:34.107151 27011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:41:34.123723 27011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:41:34.127768 27011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:41:34.128064 27011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:41:34.129088 27011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:41:34.130503 27011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:41:34.130613 27011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:34.130733 27011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:41:34.130811 27011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:34.164268 27011 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:43191
I20260810 06:41:34.164302 27220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:43191 every 8 connection(s)
I20260810 06:41:34.165834 27011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:41:34.172582 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 27011
Aug 10 06:41:34 dist-test-slave-4tdj krb5kdc[25533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344094, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:41:34.185817 26795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.168353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:42993 (local address 127.31.59.254:42153)
0810 06:41:34.168572 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:34.168578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:34.169813 (+ 1235us) server_negotiation.cc:407] Connection header received
0810 06:41:34.171686 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:34.171690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:34.171754 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:34.171839 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:34.173459 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.174570 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.176196 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.176698 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.176838 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.180036 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:34.180041 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:34.180045 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:34.180076 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:34.182779 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:34.183538 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:34.183555 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:34.183559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:34.183681 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:34.184163 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:34.184168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:34.184172 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:34.184447 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:34.184595 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:34.185240 (+ 645us) server_negotiation.cc:299] Negotiation successful
0810 06:41:34.185443 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":70}
I20260810 06:41:34.186751 27224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.168819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:42993)
0810 06:41:34.169657 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:34.169698 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:41:34.171421 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:41:34.172045 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:41:34.172052 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:41:34.172609 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:41:34.173277 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:34.173306 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.174731 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.174734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:34.176065 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:34.176078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.176907 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.176916 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:34.177191 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.177786 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:41:34.177808 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:41:34.179850 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:41:34.182899 (+ 3049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:41:34.182908 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:41:34.182923 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:41:34.183429 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:41:34.183795 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:41:34.183801 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:41:34.183805 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:41:34.184048 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:41:34.184571 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:41:34.184579 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:41:34.184893 (+ 314us) client_negotiation.cc:769] Sending connection context
0810 06:41:34.185131 (+ 238us) client_negotiation.cc:240] Negotiation successful
0810 06:41:34.185621 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":282,"threads_started":1}
I20260810 06:41:34.187952 27225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.173444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52738 (local address 127.31.59.254:42153)
0810 06:41:34.174033 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:34.174039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:34.174355 (+ 316us) server_negotiation.cc:407] Connection header received
0810 06:41:34.174551 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:34.174557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:34.174631 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:34.174737 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:34.176746 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.177584 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.179049 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.179563 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.179687 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.182392 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:34.182398 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:34.182401 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:34.182444 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:34.185069 (+ 2625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:34.185747 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:34.185753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:34.185757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:34.185906 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:34.186421 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:34.186427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:34.186431 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:34.186728 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:34.187001 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:34.187339 (+ 338us) server_negotiation.cc:299] Negotiation successful
0810 06:41:34.187574 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":307,"threads_started":1}
I20260810 06:41:34.188431 27221 heartbeater.cc:347] Connected to a master server at 127.31.59.254:42153
I20260810 06:41:34.188731 27221 heartbeater.cc:464] Registering TS with master...
I20260810 06:41:34.189465 27221 heartbeater.cc:511] Master 127.31.59.254:42153 requested a full tablet report, sending...
I20260810 06:41:34.190847 26466 ts_manager.cc:194] Registered new tserver with Master: 1d8cdef00ce34cc99e23974f6ec4a370 (127.31.59.195:43191)
I20260810 06:41:34.191668 26466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:42993
I20260810 06:41:34.202447 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:41:34.208389 27225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.203298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52748 (local address 127.31.59.254:42153)
0810 06:41:34.203444 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:34.203448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:34.203888 (+ 440us) server_negotiation.cc:407] Connection header received
0810 06:41:34.204057 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:34.204061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:34.204184 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:34.204289 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:41:34.204672 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.205317 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.206793 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.207190 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.207278 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.207699 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:41:34.207949 (+ 250us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 06:41:34.208201 (+ 252us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52748: token expired
Metrics: {"server-negotiator.queue_time_us":45}
W20260810 06:41:34.208490 27225 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52748: token expired
W20260810 06:41:34.208971 26331 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.31.59.254:42153)
I20260810 06:41:34.221601 27225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.209448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52756 (local address 127.31.59.254:42153)
0810 06:41:34.209604 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:34.209609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:34.209625 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:41:34.209776 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:34.209781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:34.209829 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:34.209904 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:34.211204 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.211899 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.213190 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.213581 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.213686 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.216523 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:34.216529 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:34.216535 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:34.216572 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:34.219292 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:34.219949 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:34.219955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:34.219959 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:34.220092 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:34.220485 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:34.220488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:34.220490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:34.220771 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:34.220940 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:34.221174 (+ 234us) server_negotiation.cc:299] Negotiation successful
0810 06:41:34.221373 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:41:34.224722 26466 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52756:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:41:34.231806 26466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:41:34.253785 27238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.242832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48968 (local address 127.31.59.193:45199)
0810 06:41:34.243384 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:34.243393 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:34.244810 (+ 1417us) server_negotiation.cc:407] Connection header received
0810 06:41:34.245206 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:34.245230 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:34.245481 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:34.245637 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:41:34.247178 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.248653 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.250876 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.252238 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.252355 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.252552 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:34.253258 (+ 706us) server_negotiation.cc:299] Negotiation successful
0810 06:41:34.253449 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":946,"threads_started":1}
I20260810 06:41:34.253826 27236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.242663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45199 (local address 127.0.0.1:48968)
0810 06:41:34.244657 (+ 1994us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:34.244710 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:41:34.244970 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:41:34.245968 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:41:34.245976 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:41:34.246023 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:41:34.246964 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:34.246977 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.248791 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.248800 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:34.250687 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:34.250702 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.252811 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.252818 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:34.253052 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.253083 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:41:34.253194 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 06:41:34.253411 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1755,"mutex_wait_us":431,"spinlock_wait_cycles":14336,"thread_start_us":2888,"threads_started":1}
I20260810 06:41:34.253860 27241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.243256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50276 (local address 127.31.59.195:43191)
0810 06:41:34.243936 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:34.243943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:34.244810 (+ 867us) server_negotiation.cc:407] Connection header received
0810 06:41:34.245206 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:34.245230 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:34.245474 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:34.245637 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:41:34.246778 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.248230 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.251179 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.252496 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.252632 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.252853 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:34.253353 (+ 500us) server_negotiation.cc:299] Negotiation successful
0810 06:41:34.253510 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":688,"threads_started":1}
I20260810 06:41:34.254673 27242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.242829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:43191 (local address 127.0.0.1:50276)
0810 06:41:34.244657 (+ 1828us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:34.244710 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:41:34.244970 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:41:34.245968 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:41:34.245976 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:41:34.246020 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:41:34.246556 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:34.246571 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.248372 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.248378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:34.251013 (+ 2635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:34.251026 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.252761 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.252767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:34.253003 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.253080 (+ 77us) client_negotiation.cc:769] Sending connection context
0810 06:41:34.253202 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 06:41:34.254265 (+ 1063us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1619,"mutex_wait_us":941,"spinlock_wait_cycles":11264,"thread_start_us":4069,"threads_started":1}
I20260810 06:41:34.254839 27243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.244533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54234 (local address 127.31.59.194:40509)
0810 06:41:34.246713 (+ 2180us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:34.246721 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:34.247426 (+ 705us) server_negotiation.cc:407] Connection header received
0810 06:41:34.247526 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:34.247533 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:34.247917 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:34.248112 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:41:34.249052 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.250504 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.252236 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.253481 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.253607 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.253801 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:34.254385 (+ 584us) server_negotiation.cc:299] Negotiation successful
0810 06:41:34.254567 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2022,"thread_start_us":1914,"threads_started":1}
I20260810 06:41:34.254863 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.243962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:40509 (local address 127.0.0.1:54234)
0810 06:41:34.247187 (+ 3225us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:34.247204 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:41:34.247310 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:41:34.248434 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:41:34.248438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:41:34.248458 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:41:34.248856 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:34.248872 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.250670 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.250675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:34.252084 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:34.252096 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.253962 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.253969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:34.254200 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.254223 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 06:41:34.254332 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 06:41:34.254463 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3102,"mutex_wait_us":26,"thread_start_us":709,"threads_started":1}
I20260810 06:41:34.258337 26684 tablet_service.cc:1467] Processing CreateTablet for tablet ed77bf52898542e9acefae6826e39604 (DEFAULT_TABLE table=test-table [id=3d06036fd47942ee906733312f533b15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:41:34.258328 26902 tablet_service.cc:1467] Processing CreateTablet for tablet ed77bf52898542e9acefae6826e39604 (DEFAULT_TABLE table=test-table [id=3d06036fd47942ee906733312f533b15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:41:34.259821 26684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed77bf52898542e9acefae6826e39604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:41:34.259821 26902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed77bf52898542e9acefae6826e39604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:41:34.260600 27112 tablet_service.cc:1467] Processing CreateTablet for tablet ed77bf52898542e9acefae6826e39604 (DEFAULT_TABLE table=test-table [id=3d06036fd47942ee906733312f533b15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:41:34.261880 27112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed77bf52898542e9acefae6826e39604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:41:34.273326 27246 tablet_bootstrap.cc:492] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370: Bootstrap starting.
I20260810 06:41:34.275202 27248 tablet_bootstrap.cc:492] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96: Bootstrap starting.
I20260810 06:41:34.276227 27250 tablet_bootstrap.cc:492] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42: Bootstrap starting.
I20260810 06:41:34.277406 27246 tablet_bootstrap.cc:654] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370: Neither blocks nor log segments found. Creating new log.
I20260810 06:41:34.279229 27248 tablet_bootstrap.cc:654] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96: Neither blocks nor log segments found. Creating new log.
I20260810 06:41:34.279312 27246 log.cc:824] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370: Log is configured to *not* fsync() on all Append() calls
I20260810 06:41:34.279579 27250 tablet_bootstrap.cc:654] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42: Neither blocks nor log segments found. Creating new log.
I20260810 06:41:34.280354 27248 log.cc:824] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96: Log is configured to *not* fsync() on all Append() calls
I20260810 06:41:34.280606 27250 log.cc:824] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42: Log is configured to *not* fsync() on all Append() calls
I20260810 06:41:34.283375 27250 tablet_bootstrap.cc:492] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42: No bootstrap required, opened a new log
I20260810 06:41:34.283460 27248 tablet_bootstrap.cc:492] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96: No bootstrap required, opened a new log
I20260810 06:41:34.283638 27250 ts_tablet_manager.cc:1397] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 06:41:34.283730 27248 ts_tablet_manager.cc:1397] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 06:41:34.288000 27246 tablet_bootstrap.cc:492] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370: No bootstrap required, opened a new log
I20260810 06:41:34.288272 27246 ts_tablet_manager.cc:1397] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260810 06:41:34.291199 27250 raft_consensus.cc:348] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } }
I20260810 06:41:34.291639 27250 raft_consensus.cc:374] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:41:34.291749 27250 raft_consensus.cc:729] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 387747fac59440b7986c72c220840f42, State: Initialized, Role: FOLLOWER
I20260810 06:41:34.292457 27250 consensus_queue.cc:260] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } }
I20260810 06:41:34.294664 27248 raft_consensus.cc:348] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } }
I20260810 06:41:34.295116 27248 raft_consensus.cc:374] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:41:34.295229 27248 raft_consensus.cc:729] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5819bbcb73394a5e879808d977b00f96, State: Initialized, Role: FOLLOWER
I20260810 06:41:34.295893 27248 consensus_queue.cc:260] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } }
I20260810 06:41:34.297928 27006 heartbeater.cc:503] Master 127.31.59.254:42153 was elected leader, sending a full tablet report...
I20260810 06:41:34.298388 27250 ts_tablet_manager.cc:1428] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260810 06:41:34.298554 27248 ts_tablet_manager.cc:1428] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
I20260810 06:41:34.298923 26792 heartbeater.cc:503] Master 127.31.59.254:42153 was elected leader, sending a full tablet report...
I20260810 06:41:34.302543 27246 raft_consensus.cc:348] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } }
I20260810 06:41:34.302958 27246 raft_consensus.cc:374] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:41:34.303081 27246 raft_consensus.cc:729] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d8cdef00ce34cc99e23974f6ec4a370, State: Initialized, Role: FOLLOWER
I20260810 06:41:34.303691 27246 consensus_queue.cc:260] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } }
I20260810 06:41:34.304607 27221 heartbeater.cc:503] Master 127.31.59.254:42153 was elected leader, sending a full tablet report...
I20260810 06:41:34.305063 27246 ts_tablet_manager.cc:1428] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370: Time spent starting tablet: real 0.017s user 0.008s sys 0.005s
I20260810 06:41:34.353329 27257 raft_consensus.cc:482] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:41:34.353654 27257 raft_consensus.cc:504] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } }
I20260810 06:41:34.355406 27257 leader_election.cc:288] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d8cdef00ce34cc99e23974f6ec4a370 (127.31.59.195:43191), 387747fac59440b7986c72c220840f42 (127.31.59.194:40509)
I20260810 06:41:34.364288 27241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.355976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:45051 (local address 127.31.59.195:43191)
0810 06:41:34.356194 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:34.356200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:34.356889 (+ 689us) server_negotiation.cc:407] Connection header received
0810 06:41:34.357019 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:34.357024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:34.357103 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:34.357188 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:41:34.357851 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.358915 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.361877 (+ 2962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.363179 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.363332 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.363456 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:34.363910 (+ 454us) server_negotiation.cc:299] Negotiation successful
0810 06:41:34.364035 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 06:41:34.364288 27243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.355973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:52831 (local address 127.31.59.194:40509)
0810 06:41:34.356366 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:34.356373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:34.356692 (+ 319us) server_negotiation.cc:407] Connection header received
0810 06:41:34.356884 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:34.356889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:34.356970 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:34.357082 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:41:34.357726 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.358842 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.361872 (+ 3030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.363197 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.363326 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.363401 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:34.363910 (+ 509us) server_negotiation.cc:299] Negotiation successful
0810 06:41:34.364036 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253}
I20260810 06:41:34.364519 27259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.355802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:43191 (local address 127.31.59.193:45051)
0810 06:41:34.356786 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:34.356801 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:41:34.356903 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:41:34.357220 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:41:34.357224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:41:34.357261 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:41:34.357699 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:34.357709 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.359015 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.359019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:34.361645 (+ 2626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:34.361662 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.363458 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.363467 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:34.363667 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.363784 (+ 117us) client_negotiation.cc:769] Sending connection context
0810 06:41:34.363930 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 06:41:34.364046 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":881,"mutex_wait_us":257,"thread_start_us":375,"threads_started":1}
I20260810 06:41:34.364674 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.355698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:40509 (local address 127.31.59.193:52831)
0810 06:41:34.356559 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:34.356582 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:41:34.356747 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:41:34.357093 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:41:34.357096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:41:34.357117 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:41:34.357581 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:34.357597 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.358995 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.359001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:34.361652 (+ 2651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:34.361662 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.363497 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.363504 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:34.363705 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.363784 (+ 79us) client_negotiation.cc:769] Sending connection context
0810 06:41:34.363881 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 06:41:34.364018 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":171,"thread_start_us":831,"threads_started":1}
I20260810 06:41:34.365654 27141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed77bf52898542e9acefae6826e39604" candidate_uuid: "5819bbcb73394a5e879808d977b00f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" is_pre_election: true
I20260810 06:41:34.365863 26935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed77bf52898542e9acefae6826e39604" candidate_uuid: "5819bbcb73394a5e879808d977b00f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "387747fac59440b7986c72c220840f42" is_pre_election: true
I20260810 06:41:34.366139 27141 raft_consensus.cc:2454] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5819bbcb73394a5e879808d977b00f96 in term 0.
I20260810 06:41:34.366348 26935 raft_consensus.cc:2454] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5819bbcb73394a5e879808d977b00f96 in term 0.
I20260810 06:41:34.366813 26607 leader_election.cc:302] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d8cdef00ce34cc99e23974f6ec4a370, 5819bbcb73394a5e879808d977b00f96; no voters:
I20260810 06:41:34.367206 27257 raft_consensus.cc:2781] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:41:34.367339 27257 raft_consensus.cc:482] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:41:34.367443 27257 raft_consensus.cc:3037] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:41:34.369562 27257 raft_consensus.cc:504] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } }
I20260810 06:41:34.370163 27257 leader_election.cc:288] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 1 election: Requested vote from peers 1d8cdef00ce34cc99e23974f6ec4a370 (127.31.59.195:43191), 387747fac59440b7986c72c220840f42 (127.31.59.194:40509)
I20260810 06:41:34.370632 27141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed77bf52898542e9acefae6826e39604" candidate_uuid: "5819bbcb73394a5e879808d977b00f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d8cdef00ce34cc99e23974f6ec4a370"
I20260810 06:41:34.370693 26935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed77bf52898542e9acefae6826e39604" candidate_uuid: "5819bbcb73394a5e879808d977b00f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "387747fac59440b7986c72c220840f42"
I20260810 06:41:34.370864 26935 raft_consensus.cc:3037] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:41:34.370844 27141 raft_consensus.cc:3037] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:41:34.374022 26935 raft_consensus.cc:2454] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5819bbcb73394a5e879808d977b00f96 in term 1.
I20260810 06:41:34.374022 27141 raft_consensus.cc:2454] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5819bbcb73394a5e879808d977b00f96 in term 1.
I20260810 06:41:34.374428 26607 leader_election.cc:302] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d8cdef00ce34cc99e23974f6ec4a370, 5819bbcb73394a5e879808d977b00f96; no voters:
I20260810 06:41:34.374688 27257 raft_consensus.cc:2781] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:41:34.375133 27257 raft_consensus.cc:686] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 LEADER]: Becoming Leader. State: Replica: 5819bbcb73394a5e879808d977b00f96, State: Running, Role: LEADER
I20260810 06:41:34.375602 27257 consensus_queue.cc:237] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } }
I20260810 06:41:34.379956 26466 catalog_manager.cc:5495] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5819bbcb73394a5e879808d977b00f96 (127.31.59.193). New cstate: current_term: 1 leader_uuid: "5819bbcb73394a5e879808d977b00f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } health_report { overall_health: UNKNOWN } } }
I20260810 06:41:34.410899 27238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.406044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48976 (local address 127.31.59.193:45199)
0810 06:41:34.406201 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:34.406206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:34.406316 (+ 110us) server_negotiation.cc:407] Connection header received
0810 06:41:34.406505 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:34.406509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:34.406591 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:34.406705 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:41:34.406982 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.407675 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.408984 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.409371 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.409455 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.409814 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:41:34.410332 (+ 518us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:41:34.410434 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:34.410592 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 06:41:34.410668 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260810 06:41:34.417490 27223 tablet.cc:2369] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:41:34.418608 27141 raft_consensus.cc:1261] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370 [term 1 FOLLOWER]: Refusing update from remote peer 5819bbcb73394a5e879808d977b00f96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:41:34.418612 26935 raft_consensus.cc:1261] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42 [term 1 FOLLOWER]: Refusing update from remote peer 5819bbcb73394a5e879808d977b00f96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:41:34.419513 27257 consensus_queue.cc:1035] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:41:34.419838 27262 consensus_queue.cc:1035] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:41:34.429625 27268 mvcc.cc:205] Tried to move back new op lower bound from 7316865410729066496 to 7316865410566135808. Current Snapshot: MvccSnapshot[applied={T|T < 7316865410729066496}]
I20260810 06:41:34.430999 27270 mvcc.cc:205] Tried to move back new op lower bound from 7316865410729066496 to 7316865410566135808. Current Snapshot: MvccSnapshot[applied={T|T < 7316865410729066496}]
I20260810 06:41:34.446273 26466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:41:34.446586 26466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:41:34.450012 26466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: Sending DeleteTablet for 3 replicas of tablet ed77bf52898542e9acefae6826e39604
I20260810 06:41:34.451107 26684 tablet_service.cc:1514] Processing DeleteTablet for tablet ed77bf52898542e9acefae6826e39604 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:41:34 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:48968
I20260810 06:41:34.451244 27112 tablet_service.cc:1514] Processing DeleteTablet for tablet ed77bf52898542e9acefae6826e39604 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:41:34 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:50276
I20260810 06:41:34.451299 26902 tablet_service.cc:1514] Processing DeleteTablet for tablet ed77bf52898542e9acefae6826e39604 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:41:34 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:54234
I20260810 06:41:34.452375 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 26537
I20260810 06:41:34.463678 27281 tablet_replica.cc:330] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370: stopping tablet replica
I20260810 06:41:34.464309 27281 raft_consensus.cc:2229] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:41:34.464809 27281 raft_consensus.cc:2258] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:41:34.465921 27282 tablet_replica.cc:330] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42: stopping tablet replica
I20260810 06:41:34.466356 27282 raft_consensus.cc:2229] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:41:34.466823 27282 raft_consensus.cc:2258] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:41:34.467123 27281 ts_tablet_manager.cc:1905] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:41:34.468771 27282 ts_tablet_manager.cc:1905] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:41:34.475350 27238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.453495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48990 (local address 127.31.59.193:45199)
0810 06:41:34.453683 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:34.453690 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:34.453709 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:41:34.453763 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:34.453769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:34.453868 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:34.453970 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:34.456077 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.457345 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.459396 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.460116 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.460237 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.466571 (+ 6334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:34.466588 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:34.466605 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:34.466650 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:34.470689 (+ 4039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:34.471685 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:34.471695 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:34.471700 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:34.471890 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:34.472372 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:34.472378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:34.472382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:34.472875 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:34.473208 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:34.474731 (+ 1523us) server_negotiation.cc:299] Negotiation successful
0810 06:41:34.474957 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 06:41:34.475740 27282 ts_tablet_manager.cc:1918] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:41:34.475900 27282 log.cc:1196] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/ed77bf52898542e9acefae6826e39604
I20260810 06:41:34.476372 26782 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:41:34.476543 27282 ts_tablet_manager.cc:1939] T ed77bf52898542e9acefae6826e39604 P 387747fac59440b7986c72c220840f42: Deleting consensus metadata
I20260810 06:41:34.478369 26438 catalog_manager.cc:4841] TS 387747fac59440b7986c72c220840f42 (127.31.59.194:40509): tablet ed77bf52898542e9acefae6826e39604 (table test-table [id=3d06036fd47942ee906733312f533b15]) successfully deleted
I20260810 06:41:34.479254 27281 ts_tablet_manager.cc:1918] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:41:34.479393 27281 log.cc:1196] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/ed77bf52898542e9acefae6826e39604
I20260810 06:41:34.479823 27281 ts_tablet_manager.cc:1939] T ed77bf52898542e9acefae6826e39604 P 1d8cdef00ce34cc99e23974f6ec4a370: Deleting consensus metadata
I20260810 06:41:34.481230 26436 catalog_manager.cc:4841] TS 1d8cdef00ce34cc99e23974f6ec4a370 (127.31.59.195:43191): tablet ed77bf52898542e9acefae6826e39604 (table test-table [id=3d06036fd47942ee906733312f533b15]) successfully deleted
I20260810 06:41:34.574133 27280 tablet_replica.cc:330] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96: stopping tablet replica
I20260810 06:41:34.575006 27280 raft_consensus.cc:2229] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:41:34.575452 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 26537
W20260810 06:41:34.599525 26439 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.59.193:45199 (error 108)
I20260810 06:41:34.599804 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 26797
I20260810 06:41:34.601337 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.600894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45199 (local address 127.0.0.1:49004)
0810 06:41:34.601071 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:34.601184 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:45199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 06:41:34.601542 26439 catalog_manager.cc:4584] TS 5819bbcb73394a5e879808d977b00f96 (127.31.59.193:45199): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ed77bf52898542e9acefae6826e39604: Network error: Client connection negotiation failed: client connection to 127.31.59.193:45199: connect: Connection refused (error 111)
I20260810 06:41:34.612864 27243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.600496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54250 (local address 127.31.59.194:40509)
0810 06:41:34.600683 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:34.600689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:34.600710 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:41:34.600874 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:34.600878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:34.600945 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:34.601062 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:34.602396 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.603122 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.604528 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.604914 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.604998 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.607599 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:34.607610 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:34.607623 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:34.607655 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:34.610493 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:34.611159 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:34.611166 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:34.611168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:34.611289 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:34.611629 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:34.611632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:34.611634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:34.611978 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:34.612206 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:34.612469 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 06:41:34.612625 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:41:34.613440 27001 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:41:34.655082 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.654619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45199 (local address 127.0.0.1:49006)
0810 06:41:34.654821 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:34.654926 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:45199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 06:41:34.656064 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.655709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45199 (local address 127.0.0.1:49016)
0810 06:41:34.655861 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:34.655945 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:45199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 06:41:34.700299 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 26797
I20260810 06:41:34.716647 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.716169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45199 (local address 127.0.0.1:49032)
0810 06:41:34.716395 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:34.716490 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:45199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 06:41:34.717597 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.717289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45199 (local address 127.0.0.1:49042)
0810 06:41:34.717407 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:34.717478 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:45199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 06:41:34.722833 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 27011
I20260810 06:41:34.735692 27241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.723604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50280 (local address 127.31.59.195:43191)
0810 06:41:34.723780 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:34.723786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:34.723806 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:41:34.723996 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:34.724000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:34.724093 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:34.724176 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:34.725508 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.726225 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.727569 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:34.727959 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:34.728079 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:34.730521 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:34.730537 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:34.730557 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:34.730591 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:34.732969 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:34.733663 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:34.733671 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:34.733674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:34.733818 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:34.734182 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:34.734186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:34.734189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:34.734569 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:34.734853 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:34.735179 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 06:41:34.735398 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:41:34.736343 27214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:41:34.798177 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.797646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45199 (local address 127.0.0.1:49048)
0810 06:41:34.797878 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:34.797992 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:45199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 06:41:34.799386 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:34.798930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45199 (local address 127.0.0.1:49064)
0810 06:41:34.799160 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:34.799250 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:45199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260810 06:41:34.823611 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 27011
I20260810 06:41:34.849810 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 26404
I20260810 06:41:34.851054 26526 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:41:34.964370 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 26404
I20260810 06:41:54.995339 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:42153
--webserver_interface=127.31.59.254
--webserver_port=43485
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:42153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:41:55.338270 27293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:41:55.338626 27293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:41:55.338774 27293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:41:55.346149 27293 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 06:41:55.346261 27293 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 06:41:55.346355 27293 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 06:41:55.346438 27293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:41:55.346537 27293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:41:55.346647 27293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:41:55.346719 27293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:41:55.346803 27293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:41:55.356820 27293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:42153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:42153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=43485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:41:55.358080 27293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:41:55.359529 27293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:41:55.371490 27300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:55.371356 27301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:55.371400 27306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:41:55.372372 27293 server_base.cc:1034] running on GCE node
I20260810 06:41:55.373097 27293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:41:55.374399 27293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:41:55.375622 27293 hybrid_clock.cc:648] HybridClock initialized: now 1786344115375560 us; error 38 us; skew 500 ppm
I20260810 06:41:55.375916 27293 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:41:55 dist-test-slave-4tdj krb5kdc[25533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344115, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:41:55.378959 27293 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:41:55.380332 27293 webserver.cc:460] Webserver started at http://127.31.59.254:43485/ using document root <none> and password file <none>
I20260810 06:41:55.380972 27293 fs_manager.cc:359] Metadata directory not provided
I20260810 06:41:55.381086 27293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:41:55.386358 27293 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:41:55.389034 27314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:55.390267 27293 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:41:55.390446 27293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "b187e4bc178e4af998b9cd688de15601"
format_stamp: "Formatted at 2026-08-10 06:41:09 on dist-test-slave-4tdj"
server_key: "1393d5f8b33cd7161f07558038fcbbd1"
server_key_iv: "bebe20a558133f48e3451e1e52e81900"
server_key_version: "encryptionkey@0"
I20260810 06:41:55.391337 27293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:41:55.408556 27293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:41:55.412420 27293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:41:55.412678 27293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:41:55.428670 27293 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:42153
I20260810 06:41:55.428707 27393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:42153 every 8 connection(s)
I20260810 06:41:55.430092 27293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:41:55.432341 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 27293
I20260810 06:41:55.433132 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:45199
--local_ip_for_outbound_sockets=127.31.59.193
--tserver_master_addrs=127.31.59.254:42153
--webserver_port=34765
--webserver_interface=127.31.59.193
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:41:55.436859 27394 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:41:55.440030 27394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: Bootstrap starting.
I20260810 06:41:55.465816 27394 log.cc:824] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: Log is configured to *not* fsync() on all Append() calls
I20260810 06:41:55.486543 27394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:41:55.487174 27394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: Bootstrap complete.
I20260810 06:41:55.500320 27394 raft_consensus.cc:348] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e4bc178e4af998b9cd688de15601" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 42153 } }
I20260810 06:41:55.501531 27394 raft_consensus.cc:729] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b187e4bc178e4af998b9cd688de15601, State: Initialized, Role: FOLLOWER
I20260810 06:41:55.502225 27394 consensus_queue.cc:260] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e4bc178e4af998b9cd688de15601" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 42153 } }
I20260810 06:41:55.502538 27394 raft_consensus.cc:388] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:41:55.502666 27394 raft_consensus.cc:482] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:41:55.502856 27394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 2 FOLLOWER]: Advancing to term 3
I20260810 06:41:55.508641 27394 raft_consensus.cc:504] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e4bc178e4af998b9cd688de15601" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 42153 } }
I20260810 06:41:55.509202 27394 leader_election.cc:302] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b187e4bc178e4af998b9cd688de15601; no voters:
I20260810 06:41:55.509855 27394 leader_election.cc:288] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 06:41:55.509992 27398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 06:41:55.511127 27398 raft_consensus.cc:686] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [term 3 LEADER]: Becoming Leader. State: Replica: b187e4bc178e4af998b9cd688de15601, State: Running, Role: LEADER
I20260810 06:41:55.511673 27398 consensus_queue.cc:237] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e4bc178e4af998b9cd688de15601" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 42153 } }
I20260810 06:41:55.511898 27394 sys_catalog.cc:564] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:41:55.513899 27402 sys_catalog.cc:455] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b187e4bc178e4af998b9cd688de15601. Latest consensus state: current_term: 3 leader_uuid: "b187e4bc178e4af998b9cd688de15601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e4bc178e4af998b9cd688de15601" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 42153 } } }
I20260810 06:41:55.514263 27402 sys_catalog.cc:458] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [sys.catalog]: This master's current role is: LEADER
I20260810 06:41:55.514780 27399 sys_catalog.cc:455] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b187e4bc178e4af998b9cd688de15601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e4bc178e4af998b9cd688de15601" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 42153 } } }
I20260810 06:41:55.515054 27399 sys_catalog.cc:458] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601 [sys.catalog]: This master's current role is: LEADER
I20260810 06:41:55.517146 27417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:41:55.525408 27417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:41:55.526882 27417 catalog_manager.cc:1259] Loaded cluster ID: b7923ee04ef04a468f78e60a359e1f21
I20260810 06:41:55.527034 27417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:41:55.530992 27417 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:41:55.532588 27417 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: Loaded TSK: 0
I20260810 06:41:55.532795 27417 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b187e4bc178e4af998b9cd688de15601: Loaded TSK: 1
F20260810 06:41:55.533946 27417 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb45df639ad google::LogMessage::Fail() at ??:0
@ 0x7fb45df6688a google::LogMessage::SendToLog() at ??:0
@ 0x7fb45df634de google::LogMessage::Flush() at ??:0
@ 0x7fb45df64a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb461cc77d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fb461cb673e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fb4739921c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fb473991783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fb473a1d2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fb473a1d0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fb473a675e9 std::function<>::operator()() at ??:0
@ 0x7fb473995cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fb473993c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb473a1b610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fb473a1b439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb460baf345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb460bb291c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fb460bb2799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb460b8deaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb463e676db start_thread at ??:0
@ 0x7fb45cdfd61f clone at ??:0
W20260810 06:41:55.838564 27396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:41:55.838819 27396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:41:55.838949 27396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:41:55.845911 27396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:41:55.846019 27396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:41:55.846174 27396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:41:55.855500 27396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:45199
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=34765
--tserver_master_addrs=127.31.59.254:42153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:41:55.856694 27396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:41:55.857924 27396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:41:55.870318 27480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:55.870313 27482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:55.870507 27488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:41:55.871124 27396 server_base.cc:1034] running on GCE node
I20260810 06:41:55.871639 27396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:41:55.872471 27396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:41:55.873695 27396 hybrid_clock.cc:648] HybridClock initialized: now 1786344115873645 us; error 43 us; skew 500 ppm
I20260810 06:41:55.874027 27396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:41:55 dist-test-slave-4tdj krb5kdc[25533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344115, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:41:55.877331 27396 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:41:55.878798 27396 webserver.cc:460] Webserver started at http://127.31.59.193:34765/ using document root <none> and password file <none>
I20260810 06:41:55.879457 27396 fs_manager.cc:359] Metadata directory not provided
I20260810 06:41:55.879601 27396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:41:55.884980 27396 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 06:41:55.887945 27499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:55.889209 27396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:41:55.889364 27396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "5819bbcb73394a5e879808d977b00f96"
format_stamp: "Formatted at 2026-08-10 06:41:10 on dist-test-slave-4tdj"
server_key: "e6f733ce508ea1f6d4a2f0606bc1079f"
server_key_iv: "f39f2747aec5611437899b58f778c691"
server_key_version: "encryptionkey@0"
I20260810 06:41:55.890272 27396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:41:55.912266 27396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:41:55.916414 27396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:41:55.916666 27396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:41:55.917670 27396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:41:55.919543 27510 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260810 06:41:55.925089 27396 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 06:41:55.925225 27396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260810 06:41:55.925347 27396 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 06:41:55.928406 27396 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 06:41:55.928484 27396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260810 06:41:55.928864 27510 tablet_bootstrap.cc:492] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96: Bootstrap starting.
I20260810 06:41:55.956310 27510 log.cc:824] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96: Log is configured to *not* fsync() on all Append() calls
I20260810 06:41:55.964133 27396 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:45199
I20260810 06:41:55.964210 27676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:45199 every 8 connection(s)
I20260810 06:41:55.965461 27396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:41:55.966498 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 27396
I20260810 06:41:55.967350 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:40509
--local_ip_for_outbound_sockets=127.31.59.194
--tserver_master_addrs=127.31.59.254:42153
--webserver_port=44731
--webserver_interface=127.31.59.194
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:41:55.969305 27510 tablet_bootstrap.cc:492] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:41:55.969944 27510 tablet_bootstrap.cc:492] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96: Bootstrap complete.
I20260810 06:41:55.971762 27510 ts_tablet_manager.cc:1397] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96: Time spent bootstrapping tablet: real 0.043s user 0.025s sys 0.016s
I20260810 06:41:55.976490 27681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:55.968535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:42299)
0810 06:41:55.974947 (+ 6412us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:55.975326 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6111,"thread_start_us":5849,"threads_started":1}
W20260810 06:41:55.977293 27677 heartbeater.cc:644] Failed to heartbeat to 127.31.59.254:42153 (0 consecutive failures): Network error: Failed to ping master at 127.31.59.254:42153: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
I20260810 06:41:55.978263 27681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:55.977881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:42329)
0810 06:41:55.978045 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:55.978130 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 06:41:55.979254 27681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:55.978858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:36967)
0810 06:41:55.978997 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:55.979104 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 06:41:55.979547 27677 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:41:55.980579 27510 raft_consensus.cc:348] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } }
I20260810 06:41:55.981732 27510 raft_consensus.cc:729] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5819bbcb73394a5e879808d977b00f96, State: Initialized, Role: FOLLOWER
I20260810 06:41:55.982386 27510 consensus_queue.cc:260] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } }
I20260810 06:41:55.983803 27681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:55.983496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:50661)
0810 06:41:55.983634 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:55.983702 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 06:41:55.984027 27510 ts_tablet_manager.cc:1428] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260810 06:41:55.984867 27681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:55.984597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:41027)
0810 06:41:55.984702 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:55.984761 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 06:41:56.299660 27682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:41:56.299922 27682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:41:56.300030 27682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:41:56.307204 27682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:41:56.307301 27682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:41:56.307415 27682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:41:56.316684 27682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:40509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=44731
--tserver_master_addrs=127.31.59.254:42153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:41:56.317840 27682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:41:56.319139 27682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:41:56.331725 27693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:56.331746 27695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:56.331771 27697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:41:56.332479 27682 server_base.cc:1034] running on GCE node
I20260810 06:41:56.333045 27682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:41:56.333906 27682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:41:56.335137 27682 hybrid_clock.cc:648] HybridClock initialized: now 1786344116335085 us; error 37 us; skew 500 ppm
I20260810 06:41:56.335450 27682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:41:56 dist-test-slave-4tdj krb5kdc[25533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344116, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:41:56.338639 27682 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:41:56.340018 27682 webserver.cc:460] Webserver started at http://127.31.59.194:44731/ using document root <none> and password file <none>
I20260810 06:41:56.340643 27682 fs_manager.cc:359] Metadata directory not provided
I20260810 06:41:56.340746 27682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:41:56.345803 27682 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:41:56.349946 27707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:56.351200 27682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260810 06:41:56.351352 27682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "387747fac59440b7986c72c220840f42"
format_stamp: "Formatted at 2026-08-10 06:41:10 on dist-test-slave-4tdj"
server_key: "f371baa3c1662cdbc4d8c8b3ec82decc"
server_key_iv: "f2af13bc6c7630f41a7cbe52b1db1bae"
server_key_version: "encryptionkey@0"
I20260810 06:41:56.352268 27682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:41:56.371291 27682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:41:56.375068 27682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:41:56.375304 27682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:41:56.376200 27682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:41:56.377429 27682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:41:56.377523 27682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:56.377629 27682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:41:56.377709 27682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:56.407819 27682 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:40509
I20260810 06:41:56.407887 27901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:40509 every 8 connection(s)
I20260810 06:41:56.409173 27682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:41:56.413645 27908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:56.411906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:53813)
0810 06:41:56.412590 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:56.412815 (+ 225us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":265,"threads_started":1}
W20260810 06:41:56.414203 27902 heartbeater.cc:644] Failed to heartbeat to 127.31.59.254:42153 (0 consecutive failures): Network error: Failed to ping master at 127.31.59.254:42153: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
I20260810 06:41:56.414939 27908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:56.414651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:45843)
0810 06:41:56.414774 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:56.414844 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 06:41:56.415609 27908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:56.415339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:37371)
0810 06:41:56.415438 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:56.415505 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 06:41:56.415800 27902 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:41:56.417712 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 27682
I20260810 06:41:56.418342 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:43191
--local_ip_for_outbound_sockets=127.31.59.195
--tserver_master_addrs=127.31.59.254:42153
--webserver_port=34839
--webserver_interface=127.31.59.195
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:41:56.736922 27910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:41:56.737188 27910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:41:56.737280 27910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:41:56.744446 27910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:41:56.744554 27910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:41:56.744676 27910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:41:56.753950 27910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:43191
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=34839
--tserver_master_addrs=127.31.59.254:42153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:41:56.755146 27910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:41:56.756448 27910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:41:56.768373 27930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:56.768415 27923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:41:56.768436 27925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:41:56.769172 27910 server_base.cc:1034] running on GCE node
I20260810 06:41:56.769694 27910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:41:56.770550 27910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:41:56.771747 27910 hybrid_clock.cc:648] HybridClock initialized: now 1786344116771692 us; error 44 us; skew 500 ppm
I20260810 06:41:56.772045 27910 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:41:56 dist-test-slave-4tdj krb5kdc[25533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344116, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:41:56.774998 27910 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:41:56.776312 27910 webserver.cc:460] Webserver started at http://127.31.59.195:34839/ using document root <none> and password file <none>
I20260810 06:41:56.776938 27910 fs_manager.cc:359] Metadata directory not provided
I20260810 06:41:56.777055 27910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:41:56.782102 27910 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:41:56.784641 27943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:56.785835 27910 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:41:56.785975 27910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "1d8cdef00ce34cc99e23974f6ec4a370"
format_stamp: "Formatted at 2026-08-10 06:41:11 on dist-test-slave-4tdj"
server_key: "9edda25b4dca6d30dbe0bea4583ba1a8"
server_key_iv: "6d24f9ed2a7a05617b33d90b3232fbc8"
server_key_version: "encryptionkey@0"
I20260810 06:41:56.786942 27910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:41:56.800724 27910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:41:56.804210 27910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:41:56.804488 27910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:41:56.805374 27910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:41:56.806641 27910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:41:56.806736 27910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:56.806838 27910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:41:56.806907 27910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:41:56.837193 27910 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:43191
I20260810 06:41:56.837248 28124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:43191 every 8 connection(s)
I20260810 06:41:56.838685 27910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:41:56.843204 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 27910
I20260810 06:41:56.843501 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:56.841611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:55839)
0810 06:41:56.842372 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:56.842620 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":276,"threads_started":1}
W20260810 06:41:56.844228 28125 heartbeater.cc:644] Failed to heartbeat to 127.31.59.254:42153 (0 consecutive failures): Network error: Failed to ping master at 127.31.59.254:42153: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
I20260810 06:41:56.845144 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:56.844801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:40061)
0810 06:41:56.844958 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:56.845028 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 06:41:56.845940 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:56.845630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:37087)
0810 06:41:56.845766 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:56.845836 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 06:41:56.846158 28125 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:41:56.986461 28133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:56.985714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:47215)
0810 06:41:56.986160 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:56.986244 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":215,"threads_started":1}
I20260810 06:41:57.417060 28135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:57.416297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:52197)
0810 06:41:57.416740 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:57.416845 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":206,"threads_started":1}
I20260810 06:41:57.766700 28137 raft_consensus.cc:482] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:41:57.766992 28137 raft_consensus.cc:504] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } }
I20260810 06:41:57.768592 28137 leader_election.cc:288] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d8cdef00ce34cc99e23974f6ec4a370 (127.31.59.195:43191), 387747fac59440b7986c72c220840f42 (127.31.59.194:40509)
Aug 10 06:41:57 dist-test-slave-4tdj krb5kdc[25533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344115, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.195@KRBTEST.COM
Aug 10 06:41:57 dist-test-slave-4tdj krb5kdc[25533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344115, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.194@KRBTEST.COM
I20260810 06:41:57.804243 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:57.768874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:40509 (local address 127.31.59.193:43371)
0810 06:41:57.779728 (+ 10854us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:57.779769 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:41:57.780384 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:41:57.785166 (+ 4782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:41:57.785171 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:41:57.785741 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:41:57.786090 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:57.786103 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:57.788228 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:57.788232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:57.789537 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:57.789577 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:57.790872 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:57.790876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:57.790985 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:57.793801 (+ 2816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:41:57.793868 (+ 67us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:41:57.796974 (+ 3106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:41:57.801510 (+ 4536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:41:57.801513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:41:57.801515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:41:57.801866 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:41:57.802310 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:41:57.802316 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:41:57.802320 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:41:57.802467 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:41:57.803202 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:41:57.803207 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:41:57.803344 (+ 137us) client_negotiation.cc:769] Sending connection context
0810 06:41:57.803449 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 06:41:57.803803 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10737,"spinlock_wait_cycles":18176,"thread_start_us":10599,"threads_started":1}
I20260810 06:41:57.804376 28139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:57.768912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:43191 (local address 127.31.59.193:60845)
0810 06:41:57.780494 (+ 11582us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:57.780510 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:41:57.780622 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:41:57.783001 (+ 2379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:41:57.783014 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:41:57.783899 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:41:57.784764 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:57.784792 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:57.786824 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:57.786830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:57.788576 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:41:57.788590 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:41:57.789885 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:41:57.789891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:41:57.790136 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:57.793799 (+ 3663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:41:57.793829 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:41:57.796628 (+ 2799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:41:57.800950 (+ 4322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:41:57.800957 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:41:57.800968 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:41:57.801381 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:41:57.801811 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:41:57.801817 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:41:57.801820 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:41:57.801997 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:41:57.802805 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:41:57.802813 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:41:57.803093 (+ 280us) client_negotiation.cc:769] Sending connection context
0810 06:41:57.803328 (+ 235us) client_negotiation.cc:240] Negotiation successful
0810 06:41:57.803995 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11476,"mutex_wait_us":178,"thread_start_us":11243,"threads_started":1}
I20260810 06:41:57.804497 28143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:57.769248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:43371 (local address 127.31.59.194:40509)
0810 06:41:57.780124 (+ 10876us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:57.780137 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:57.780185 (+ 48us) server_negotiation.cc:407] Connection header received
0810 06:41:57.784405 (+ 4220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:57.784417 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:57.784852 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:57.785140 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:57.786259 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:57.788068 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:57.789729 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:57.790425 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:57.790891 (+ 466us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:57.797210 (+ 6319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:57.797217 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:57.797227 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:57.797262 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:57.800796 (+ 3534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:57.802001 (+ 1205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:57.802008 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:57.802013 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:57.802171 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:57.802588 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:57.802591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:57.802594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:57.803070 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:57.803227 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:57.803735 (+ 508us) server_negotiation.cc:299] Negotiation successful
0810 06:41:57.804140 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10687,"thread_start_us":10527,"threads_started":1}
I20260810 06:41:57.804880 28142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:57.769137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:60845 (local address 127.31.59.195:43191)
0810 06:41:57.779429 (+ 10292us) server_negotiation.cc:206] Beginning negotiation
0810 06:41:57.779447 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 06:41:57.780628 (+ 1181us) server_negotiation.cc:407] Connection header received
0810 06:41:57.782108 (+ 1480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:41:57.782117 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:41:57.782538 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:41:57.782819 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:41:57.784951 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:57.786618 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:57.788714 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:41:57.789711 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:41:57.789871 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:41:57.796942 (+ 7071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:41:57.796952 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:41:57.796970 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:41:57.797030 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:41:57.800796 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:57.801489 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:57.801495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:57.801501 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:57.801683 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:41:57.802104 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:41:57.802109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:41:57.802113 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:41:57.802656 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:41:57.802821 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 06:41:57.803569 (+ 748us) server_negotiation.cc:299] Negotiation successful
0810 06:41:57.803967 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10115,"thread_start_us":9985,"threads_started":1}
W20260810 06:41:57.808072 27504 leader_election.cc:341] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 387747fac59440b7986c72c220840f42 (127.31.59.194:40509): Not found: Tablet not found: ed77bf52898542e9acefae6826e39604
W20260810 06:41:57.808350 27502 leader_election.cc:341] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d8cdef00ce34cc99e23974f6ec4a370 (127.31.59.195:43191): Not found: Tablet not found: ed77bf52898542e9acefae6826e39604
I20260810 06:41:57.808491 27502 leader_election.cc:302] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5819bbcb73394a5e879808d977b00f96; no voters: 1d8cdef00ce34cc99e23974f6ec4a370, 387747fac59440b7986c72c220840f42
I20260810 06:41:57.808928 28137 raft_consensus.cc:2726] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:41:57.847438 28144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:57.846629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:45031)
0810 06:41:57.847078 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:57.847213 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":214,"threads_started":1}
I20260810 06:41:57.987738 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:57.987321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:35463)
0810 06:41:57.987467 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:57.987592 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 06:41:58.418609 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:58.417888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:59097)
0810 06:41:58.418348 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:58.418429 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":243,"threads_started":1}
I20260810 06:41:58.848927 28147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:58.848283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:54363)
0810 06:41:58.848697 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:58.848774 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":207,"threads_started":1}
I20260810 06:41:58.989390 28148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:58.988565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:42345)
0810 06:41:58.989073 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:58.989163 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":227,"threads_started":1}
I20260810 06:41:59.420205 28150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:59.419519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:32899)
0810 06:41:59.419961 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:59.420042 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":241,"threads_started":1}
I20260810 06:41:59.630920 28151 raft_consensus.cc:482] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:41:59.631115 28151 raft_consensus.cc:504] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } }
I20260810 06:41:59.631785 28151 leader_election.cc:288] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d8cdef00ce34cc99e23974f6ec4a370 (127.31.59.195:43191), 387747fac59440b7986c72c220840f42 (127.31.59.194:40509)
W20260810 06:41:59.632831 27504 leader_election.cc:341] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 387747fac59440b7986c72c220840f42 (127.31.59.194:40509): Not found: Tablet not found: ed77bf52898542e9acefae6826e39604
W20260810 06:41:59.633009 27502 leader_election.cc:341] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d8cdef00ce34cc99e23974f6ec4a370 (127.31.59.195:43191): Not found: Tablet not found: ed77bf52898542e9acefae6826e39604
I20260810 06:41:59.633121 27502 leader_election.cc:302] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5819bbcb73394a5e879808d977b00f96; no voters: 1d8cdef00ce34cc99e23974f6ec4a370, 387747fac59440b7986c72c220840f42
I20260810 06:41:59.633335 28151 raft_consensus.cc:2726] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:41:59.850592 28153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:59.849803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:39385)
0810 06:41:59.850338 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:59.850426 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":280,"threads_started":1}
I20260810 06:41:59.991305 28157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:41:59.990404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:33915)
0810 06:41:59.990953 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0810 06:41:59.991086 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":283,"threads_started":1}
I20260810 06:42:00.421933 28158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:00.421134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:52503)
0810 06:42:00.421656 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:00.421754 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":224,"threads_started":1}
I20260810 06:42:00.852109 28159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:00.851394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:46923)
0810 06:42:00.851823 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:00.851904 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":205,"threads_started":1}
I20260810 06:42:00.993067 28160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:00.992284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:58307)
0810 06:42:00.992760 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:00.992843 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":266,"threads_started":1}
I20260810 06:42:01.423672 28161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:01.422937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:33611)
0810 06:42:01.423381 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:01.423469 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":238,"threads_started":1}
I20260810 06:42:01.853627 28162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:01.852963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:59849)
0810 06:42:01.853375 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:01.853454 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":223,"threads_started":1}
I20260810 06:42:01.994719 28163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:01.993918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:40611)
0810 06:42:01.994418 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:01.994535 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":238,"threads_started":1}
I20260810 06:42:02.425259 28164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:02.424491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:41721)
0810 06:42:02.424968 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:02.425056 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":251,"threads_started":1}
I20260810 06:42:02.671885 28165 raft_consensus.cc:482] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:42:02.672106 28165 raft_consensus.cc:504] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } }
I20260810 06:42:02.672907 28165 leader_election.cc:288] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d8cdef00ce34cc99e23974f6ec4a370 (127.31.59.195:43191), 387747fac59440b7986c72c220840f42 (127.31.59.194:40509)
W20260810 06:42:02.673848 27502 leader_election.cc:341] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d8cdef00ce34cc99e23974f6ec4a370 (127.31.59.195:43191): Not found: Tablet not found: ed77bf52898542e9acefae6826e39604
W20260810 06:42:02.674037 27504 leader_election.cc:341] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 387747fac59440b7986c72c220840f42 (127.31.59.194:40509): Not found: Tablet not found: ed77bf52898542e9acefae6826e39604
I20260810 06:42:02.674221 27504 leader_election.cc:302] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5819bbcb73394a5e879808d977b00f96; no voters: 1d8cdef00ce34cc99e23974f6ec4a370, 387747fac59440b7986c72c220840f42
I20260810 06:42:02.674540 28165 raft_consensus.cc:2726] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:42:02.855237 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:02.854489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:53113)
0810 06:42:02.854959 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:02.855063 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":260,"threads_started":1}
I20260810 06:42:02.996322 28167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:02.995517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:52721)
0810 06:42:02.995999 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:02.996102 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":213,"threads_started":1}
I20260810 06:42:03.426908 28168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:03.426117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:49425)
0810 06:42:03.426638 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:03.426730 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":233,"threads_started":1}
I20260810 06:42:03.857007 28169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:03.856139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:46705)
0810 06:42:03.856739 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:03.856826 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":241,"threads_started":1}
I20260810 06:42:03.997807 28170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:03.997142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:44617)
0810 06:42:03.997557 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:03.997647 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":211,"threads_started":1}
I20260810 06:42:04.428392 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:04.427683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:51247)
0810 06:42:04.428124 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:04.428216 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":211,"threads_started":1}
I20260810 06:42:04.858732 28172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:04.857985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:45033)
0810 06:42:04.858435 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:04.858543 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":212,"threads_started":1}
I20260810 06:42:04.999373 28173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:04.998685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:45185)
0810 06:42:04.999108 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:04.999189 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":215,"threads_started":1}
I20260810 06:42:05.429854 28175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:05.429188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:35711)
0810 06:42:05.429615 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:05.429701 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":209,"threads_started":1}
I20260810 06:42:05.860328 28177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:05.859579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:54387)
0810 06:42:05.860068 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:05.860152 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":222,"threads_started":1}
I20260810 06:42:06.001030 28179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:06.000305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:35329)
0810 06:42:06.000773 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:06.000860 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":258,"threads_started":1}
I20260810 06:42:06.431442 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:06.430730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:42141)
0810 06:42:06.431171 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:06.431262 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":219,"threads_started":1}
I20260810 06:42:06.537684 28181 raft_consensus.cc:482] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:42:06.537874 28181 raft_consensus.cc:504] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } }
I20260810 06:42:06.538554 28181 leader_election.cc:288] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d8cdef00ce34cc99e23974f6ec4a370 (127.31.59.195:43191), 387747fac59440b7986c72c220840f42 (127.31.59.194:40509)
W20260810 06:42:06.539556 27504 leader_election.cc:341] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 387747fac59440b7986c72c220840f42 (127.31.59.194:40509): Not found: Tablet not found: ed77bf52898542e9acefae6826e39604
W20260810 06:42:06.539726 27502 leader_election.cc:341] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d8cdef00ce34cc99e23974f6ec4a370 (127.31.59.195:43191): Not found: Tablet not found: ed77bf52898542e9acefae6826e39604
I20260810 06:42:06.539840 27502 leader_election.cc:302] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5819bbcb73394a5e879808d977b00f96; no voters: 1d8cdef00ce34cc99e23974f6ec4a370, 387747fac59440b7986c72c220840f42
I20260810 06:42:06.540050 28181 raft_consensus.cc:2726] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:42:06.861914 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:06.861244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:55483)
0810 06:42:06.861655 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:06.861736 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":211,"threads_started":1}
I20260810 06:42:07.002866 28183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:07.001946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:57301)
0810 06:42:07.002556 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:07.002649 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":270,"threads_started":1}
I20260810 06:42:07.432999 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:07.432260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:40391)
0810 06:42:07.432755 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:07.432835 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":234,"threads_started":1}
I20260810 06:42:07.863440 28185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:07.862763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:42573)
0810 06:42:07.863193 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:07.863278 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":209,"threads_started":1}
I20260810 06:42:08.004416 28186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:08.003704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:43767)
0810 06:42:08.004166 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:08.004251 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":252,"threads_started":1}
I20260810 06:42:08.434616 28187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:08.433828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:35495)
0810 06:42:08.434312 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:08.434422 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":250,"threads_started":1}
I20260810 06:42:08.865046 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:08.864318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:45533)
0810 06:42:08.864790 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:08.864872 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":239,"threads_started":1}
I20260810 06:42:09.006037 28189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:09.005316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:45533)
0810 06:42:09.005756 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:09.005835 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":221,"threads_started":1}
I20260810 06:42:09.384719 28190 raft_consensus.cc:482] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:42:09.384965 28190 raft_consensus.cc:504] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5819bbcb73394a5e879808d977b00f96" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45199 } } peers { permanent_uuid: "1d8cdef00ce34cc99e23974f6ec4a370" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 43191 } } peers { permanent_uuid: "387747fac59440b7986c72c220840f42" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 40509 } }
I20260810 06:42:09.385697 28190 leader_election.cc:288] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d8cdef00ce34cc99e23974f6ec4a370 (127.31.59.195:43191), 387747fac59440b7986c72c220840f42 (127.31.59.194:40509)
W20260810 06:42:09.386602 27502 leader_election.cc:341] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d8cdef00ce34cc99e23974f6ec4a370 (127.31.59.195:43191): Not found: Tablet not found: ed77bf52898542e9acefae6826e39604
W20260810 06:42:09.386770 27504 leader_election.cc:341] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 387747fac59440b7986c72c220840f42 (127.31.59.194:40509): Not found: Tablet not found: ed77bf52898542e9acefae6826e39604
I20260810 06:42:09.386915 27504 leader_election.cc:302] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5819bbcb73394a5e879808d977b00f96; no voters: 1d8cdef00ce34cc99e23974f6ec4a370, 387747fac59440b7986c72c220840f42
I20260810 06:42:09.387151 28190 raft_consensus.cc:2726] T ed77bf52898542e9acefae6826e39604 P 5819bbcb73394a5e879808d977b00f96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:42:09.436225 28191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:09.435476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:35593)
0810 06:42:09.435941 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:09.436049 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":239,"threads_started":1}
I20260810 06:42:09.866698 28192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:09.865938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:39481)
0810 06:42:09.866430 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:09.866527 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":239,"threads_started":1}
I20260810 06:42:10.008002 28193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:10.007029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:47545)
0810 06:42:10.007574 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:10.007711 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":276,"threads_started":1}
I20260810 06:42:10.437826 28194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:10.437076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:58611)
0810 06:42:10.437546 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:10.437631 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":227,"threads_started":1}
I20260810 06:42:10.868294 28195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:10.867540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:43113)
0810 06:42:10.868003 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:10.868088 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":219,"threads_started":1}
I20260810 06:42:11.009953 28198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:11.009211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.193:47221)
0810 06:42:11.009698 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:11.009781 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":234,"threads_started":1}
I20260810 06:42:11.439633 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:11.438888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.194:47295)
0810 06:42:11.439360 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:11.439454 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":224,"threads_started":1}
I20260810 06:42:11.846980 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 27396
I20260810 06:42:11.861677 28203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:11.847968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37572 (local address 127.31.59.193:45199)
0810 06:42:11.848546 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:11.848554 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:11.848580 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:42:11.848641 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:11.848649 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:11.848869 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:11.849013 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:11.850804 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:11.851781 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:11.853044 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:11.853665 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:11.853780 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:11.856396 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:11.856408 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:11.856422 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:11.856455 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:11.859069 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:11.859677 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:11.859684 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:11.859686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:11.859841 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:11.860198 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:11.860202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:11.860204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:11.860555 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:11.860835 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:11.861133 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 06:42:11.861386 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":286,"threads_started":1}
I20260810 06:42:11.862949 27670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:11.870092 28205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:11.869349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:42153 (local address 127.31.59.195:41763)
0810 06:42:11.869823 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:11.869944 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.254:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":261,"threads_started":1}
I20260810 06:42:11.951354 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 27396
I20260810 06:42:11.977182 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 27682
I20260810 06:42:11.990402 28206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:11.977856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37720 (local address 127.31.59.194:40509)
0810 06:42:11.978409 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:11.978413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:11.978430 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:42:11.978486 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:11.978489 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:11.978550 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:11.978637 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:11.979979 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:11.980822 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:11.982072 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:11.982587 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:11.982688 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:11.985305 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:11.985310 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:11.985314 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:11.985350 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:11.987845 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:11.988521 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:11.988526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:11.988528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:11.988674 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:11.989019 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:11.989022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:11.989024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:11.989286 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:11.989482 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:11.989789 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 06:42:11.989967 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":310,"threads_started":1}
I20260810 06:42:11.991104 27890 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:12.073350 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 27682
I20260810 06:42:12.095378 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 27910
I20260810 06:42:12.113036 28208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:12.096118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58064 (local address 127.31.59.195:43191)
0810 06:42:12.096649 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:12.096654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:12.096678 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:42:12.096736 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:12.096740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:12.096817 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:12.096917 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:12.098238 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:12.099045 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:12.100321 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:12.100751 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:12.100843 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:12.107891 (+ 7048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:12.107896 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:12.107899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:12.107933 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:12.110536 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:12.111178 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:12.111182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:12.111185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:12.111329 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:12.111677 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:12.111681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:12.111683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:12.111947 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:12.112156 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:12.112548 (+ 392us) server_negotiation.cc:299] Negotiation successful
0810 06:42:12.112750 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":257,"threads_started":1}
I20260810 06:42:12.113770 28113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:12.196338 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 27910
2026-08-10T06:42:12Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65130 ms)
[----------] 30 tests from SecurityITest (300375 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:42:12 dist-test-slave-4tdj krb5kdc[28213](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:42:12 dist-test-slave-4tdj krb5kdc[28213](info): set up 2 sockets
Aug 10 06:42:12 dist-test-slave-4tdj krb5kdc[28213](info): commencing operation
krb5kdc: starting...
W20260810 06:42:13.443820 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.005s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:42:13 dist-test-slave-4tdj krb5kdc[28213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:42:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:42:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:42:13.701457 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:41553
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:41553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:14.032557 28240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:14.032829 28240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:14.032963 28240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:14.040069 28240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:42:14.040167 28240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:14.040241 28240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:14.040318 28240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:42:14.040375 28240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:42:14.049974 28240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:41553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:41553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:14.051213 28240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:14.052460 28240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:14.063898 28248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:14.063936 28252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:14.064179 28245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:14.065179 28240 server_base.cc:1034] running on GCE node
I20260810 06:42:14.065955 28240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:14.067380 28240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:14.068617 28240 hybrid_clock.cc:648] HybridClock initialized: now 1786344134068571 us; error 27 us; skew 500 ppm
I20260810 06:42:14.068959 28240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:14 dist-test-slave-4tdj krb5kdc[28213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344134, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:14.072037 28240 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:42:14.073457 28240 webserver.cc:460] Webserver started at http://127.31.59.254:38137/ using document root <none> and password file <none>
I20260810 06:42:14.074115 28240 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:14.074236 28240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:14.074537 28240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:14.077454 28240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "0903377353424db2ad1c67b6d26dc04e"
format_stamp: "Formatted at 2026-08-10 06:42:14 on dist-test-slave-4tdj"
server_key: "e9845db488bbe82b1d9fb28c08b62c6d"
server_key_iv: "d6c91a5c9c2a30fe1936053d5042eede"
server_key_version: "encryptionkey@0"
I20260810 06:42:14.078178 28240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "0903377353424db2ad1c67b6d26dc04e"
format_stamp: "Formatted at 2026-08-10 06:42:14 on dist-test-slave-4tdj"
server_key: "e9845db488bbe82b1d9fb28c08b62c6d"
server_key_iv: "d6c91a5c9c2a30fe1936053d5042eede"
server_key_version: "encryptionkey@0"
I20260810 06:42:14.083276 28240 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260810 06:42:14.086669 28268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:14.087997 28240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 06:42:14.088168 28240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "0903377353424db2ad1c67b6d26dc04e"
format_stamp: "Formatted at 2026-08-10 06:42:14 on dist-test-slave-4tdj"
server_key: "e9845db488bbe82b1d9fb28c08b62c6d"
server_key_iv: "d6c91a5c9c2a30fe1936053d5042eede"
server_key_version: "encryptionkey@0"
I20260810 06:42:14.088346 28240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:14.105691 28240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:14.110016 28240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:14.110312 28240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:14.128469 28240 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:41553
I20260810 06:42:14.128517 28348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:41553 every 8 connection(s)
I20260810 06:42:14.129818 28240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:42:14.133570 28349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:14.136837 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 28240
I20260810 06:42:14.137033 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:42:14.137720 31983 external_mini_cluster.cc:1428] Setting key c3ae779ea291c20137b598a6229c0647
I20260810 06:42:14.146960 28349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e: Bootstrap starting.
I20260810 06:42:14.150557 28349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:14.151561 28349 log.cc:824] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e: Log is configured to *not* fsync() on all Append() calls
Aug 10 06:42:14 dist-test-slave-4tdj krb5kdc[28213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:42:14.154533 28349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e: No bootstrap required, opened a new log
I20260810 06:42:14.161860 28353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:14.139696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48752 (local address 127.31.59.254:41553)
0810 06:42:14.140537 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:14.140553 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:14.140606 (+ 53us) server_negotiation.cc:407] Connection header received
0810 06:42:14.142714 (+ 2108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:14.142745 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:14.143306 (+ 561us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:14.143759 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:14.145085 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:14.147046 (+ 1961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:14.148402 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:14.149400 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:14.149580 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:14.153550 (+ 3970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:14.153560 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:14.153579 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:14.153628 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:14.157501 (+ 3873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:14.158178 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:14.158186 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:14.158195 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:14.158417 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:14.158764 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:14.158769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:14.158772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:14.159436 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:14.159730 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:14.160089 (+ 359us) server_negotiation.cc:299] Negotiation successful
0810 06:42:14.160522 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":356,"threads_started":1}
I20260810 06:42:14.164318 28349 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0903377353424db2ad1c67b6d26dc04e" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 41553 } }
I20260810 06:42:14.164669 28349 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:14.164768 28349 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0903377353424db2ad1c67b6d26dc04e, State: Initialized, Role: FOLLOWER
I20260810 06:42:14.165331 28349 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0903377353424db2ad1c67b6d26dc04e" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 41553 } }
I20260810 06:42:14.165575 28349 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:42:14.165699 28349 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:42:14.165853 28349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:14.167960 28349 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0903377353424db2ad1c67b6d26dc04e" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 41553 } }
I20260810 06:42:14.168483 28349 leader_election.cc:302] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0903377353424db2ad1c67b6d26dc04e; no voters:
I20260810 06:42:14.169073 28349 leader_election.cc:288] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:42:14.169193 28357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:14.169803 28357 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e [term 1 LEADER]: Becoming Leader. State: Replica: 0903377353424db2ad1c67b6d26dc04e, State: Running, Role: LEADER
I20260810 06:42:14.170408 28357 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0903377353424db2ad1c67b6d26dc04e" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 41553 } }
I20260810 06:42:14.170982 28349 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:42:14.173204 28358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0903377353424db2ad1c67b6d26dc04e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0903377353424db2ad1c67b6d26dc04e" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 41553 } } }
I20260810 06:42:14.173856 28358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:14.173182 28359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0903377353424db2ad1c67b6d26dc04e. Latest consensus state: current_term: 1 leader_uuid: "0903377353424db2ad1c67b6d26dc04e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0903377353424db2ad1c67b6d26dc04e" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 41553 } } }
I20260810 06:42:14.174566 28359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:14.175457 28365 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:42:14.180634 28365 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:42:14.188769 28365 catalog_manager.cc:1347] Generated new cluster ID: 417cef4c801d4ceba96b2bc11307d37c
I20260810 06:42:14.188870 28365 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:42:14.202759 28365 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:42:14.204017 28365 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:42:14.214919 28365 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e: Generated new TSK 0
I20260810 06:42:14.215679 28365 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
I20260810 06:42:14.287046 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:41553
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:14.618475 28387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:14.618729 28387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:14.618871 28387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:14.625878 28387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:14.625979 28387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:14.626132 28387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:42:14.635488 28387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:41553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:14.636693 28387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:14.637923 28387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:14.650668 28395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:14.650694 28394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:14.650635 28399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:14.651512 28387 server_base.cc:1034] running on GCE node
I20260810 06:42:14.652113 28387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:14.652984 28387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:14.654177 28387 hybrid_clock.cc:648] HybridClock initialized: now 1786344134654131 us; error 37 us; skew 500 ppm
I20260810 06:42:14.654486 28387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:14 dist-test-slave-4tdj krb5kdc[28213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344134, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:14.657490 28387 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:42:14.658874 28387 webserver.cc:460] Webserver started at http://127.31.59.193:38965/ using document root <none> and password file <none>
I20260810 06:42:14.659529 28387 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:14.659631 28387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:14.659891 28387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:14.662716 28387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "606e53a085bb4d34af1ae6981f403225"
format_stamp: "Formatted at 2026-08-10 06:42:14 on dist-test-slave-4tdj"
server_key: "cb8c3dd52ca34c8d0f2d81c7874fa8c9"
server_key_iv: "93eb0efe212282d6d94ba523c3e281be"
server_key_version: "encryptionkey@0"
I20260810 06:42:14.663415 28387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "606e53a085bb4d34af1ae6981f403225"
format_stamp: "Formatted at 2026-08-10 06:42:14 on dist-test-slave-4tdj"
server_key: "cb8c3dd52ca34c8d0f2d81c7874fa8c9"
server_key_iv: "93eb0efe212282d6d94ba523c3e281be"
server_key_version: "encryptionkey@0"
I20260810 06:42:14.668206 28387 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:42:14.671501 28414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:14.672788 28387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 06:42:14.672936 28387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "606e53a085bb4d34af1ae6981f403225"
format_stamp: "Formatted at 2026-08-10 06:42:14 on dist-test-slave-4tdj"
server_key: "cb8c3dd52ca34c8d0f2d81c7874fa8c9"
server_key_iv: "93eb0efe212282d6d94ba523c3e281be"
server_key_version: "encryptionkey@0"
I20260810 06:42:14.673118 28387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:14.692220 28387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:14.696151 28387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:14.696405 28387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:14.697306 28387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:42:14.698617 28387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:42:14.698697 28387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:14.698843 28387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:42:14.698899 28387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:14.729231 28387 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:34309
I20260810 06:42:14.729313 28599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:34309 every 8 connection(s)
I20260810 06:42:14.730561 28387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:42:14.732704 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 28387
I20260810 06:42:14.732908 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:42:14.733677 31983 external_mini_cluster.cc:1428] Setting key e1a617ff068966a72507abedad6582e3
Aug 10 06:42:14 dist-test-slave-4tdj krb5kdc[28213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344134, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:42:14.749908 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:14.732972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:57539 (local address 127.31.59.254:41553)
0810 06:42:14.733471 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:14.733477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:14.734551 (+ 1074us) server_negotiation.cc:407] Connection header received
0810 06:42:14.736347 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:14.736353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:14.736439 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:14.736546 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:14.737987 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:14.738980 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:14.740536 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:14.741049 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:14.741178 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:14.744523 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:14.744528 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:14.744531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:14.744564 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:14.747056 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:14.747710 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:14.747716 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:14.747720 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:14.747870 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:14.748259 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:14.748262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:14.748264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:14.748520 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:14.748663 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:14.749329 (+ 666us) server_negotiation.cc:299] Negotiation successful
0810 06:42:14.749563 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":231,"threads_started":1}
I20260810 06:42:14.750828 28603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:14.733539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:41553 (local address 127.31.59.193:57539)
0810 06:42:14.734373 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:14.734425 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:14.736084 (+ 1659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:14.736741 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:14.736749 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:14.737280 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:42:14.737822 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:14.737840 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:14.739167 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:14.739172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:14.740392 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:14.740405 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:14.741298 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:14.741303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:14.741497 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:14.742119 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:42:14.742145 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:42:14.744312 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:42:14.747187 (+ 2875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:42:14.747194 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:42:14.747205 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:42:14.747602 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:42:14.747965 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:42:14.747968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:42:14.747970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:42:14.748147 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:42:14.748651 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:42:14.748658 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:42:14.748955 (+ 297us) client_negotiation.cc:769] Sending connection context
0810 06:42:14.749206 (+ 251us) client_negotiation.cc:240] Negotiation successful
0810 06:42:14.749680 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":312,"threads_started":1}
I20260810 06:42:14.752453 28600 heartbeater.cc:347] Connected to a master server at 127.31.59.254:41553
I20260810 06:42:14.752753 28600 heartbeater.cc:464] Registering TS with master...
I20260810 06:42:14.753409 28600 heartbeater.cc:511] Master 127.31.59.254:41553 requested a full tablet report, sending...
I20260810 06:42:14.755493 28292 ts_manager.cc:194] Registered new tserver with Master: 606e53a085bb4d34af1ae6981f403225 (127.31.59.193:34309)
I20260810 06:42:14.756932 28292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:57539
WARNING: no policy specified for kudu/127.31.59.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
I20260810 06:42:14.795140 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:41553
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:15.134213 28606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:15.134531 28606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:15.134692 28606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:15.141937 28606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:15.142071 28606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:15.142243 28606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:42:15.152181 28606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:41553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:15.153478 28606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:15.154870 28606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:15.167949 28616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:15.168020 28617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:15.167971 28621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:15.169044 28606 server_base.cc:1034] running on GCE node
I20260810 06:42:15.169624 28606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:15.170557 28606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:15.171792 28606 hybrid_clock.cc:648] HybridClock initialized: now 1786344135171725 us; error 54 us; skew 500 ppm
I20260810 06:42:15.172181 28606 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:15 dist-test-slave-4tdj krb5kdc[28213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344135, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:15.175405 28606 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:42:15.176898 28606 webserver.cc:460] Webserver started at http://127.31.59.194:33361/ using document root <none> and password file <none>
I20260810 06:42:15.177624 28606 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:15.177744 28606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:15.178049 28606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:15.180892 28606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "f9b2817ec71e4b4dbde231bcdb7fa506"
format_stamp: "Formatted at 2026-08-10 06:42:15 on dist-test-slave-4tdj"
server_key: "dda8adcc186ffd82c6de2f850d033ad4"
server_key_iv: "313475b6ccde3aafaabfe5c7fc6db67e"
server_key_version: "encryptionkey@0"
I20260810 06:42:15.181619 28606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "f9b2817ec71e4b4dbde231bcdb7fa506"
format_stamp: "Formatted at 2026-08-10 06:42:15 on dist-test-slave-4tdj"
server_key: "dda8adcc186ffd82c6de2f850d033ad4"
server_key_iv: "313475b6ccde3aafaabfe5c7fc6db67e"
server_key_version: "encryptionkey@0"
I20260810 06:42:15.186502 28606 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 06:42:15.189700 28635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:15.190955 28606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:42:15.191112 28606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "f9b2817ec71e4b4dbde231bcdb7fa506"
format_stamp: "Formatted at 2026-08-10 06:42:15 on dist-test-slave-4tdj"
server_key: "dda8adcc186ffd82c6de2f850d033ad4"
server_key_iv: "313475b6ccde3aafaabfe5c7fc6db67e"
server_key_version: "encryptionkey@0"
I20260810 06:42:15.191283 28606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:15.218176 28606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:15.222153 28606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:15.222451 28606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:15.223418 28606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:42:15.224756 28606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:42:15.224828 28606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:15.224939 28606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:42:15.224989 28606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:15.255759 28606 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:33705
I20260810 06:42:15.255810 28815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:33705 every 8 connection(s)
I20260810 06:42:15.256969 28606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:42:15.260798 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 28606
I20260810 06:42:15.260995 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:42:15.261499 31983 external_mini_cluster.cc:1428] Setting key f78287e63245d7a8ecf405af272910fe
Aug 10 06:42:15 dist-test-slave-4tdj krb5kdc[28213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344135, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:42:15.277271 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:15.259423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:33205 (local address 127.31.59.254:41553)
0810 06:42:15.259876 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:15.259882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:15.260989 (+ 1107us) server_negotiation.cc:407] Connection header received
0810 06:42:15.262928 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:15.262934 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:15.263018 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:15.263121 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:15.264643 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.265681 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.267237 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.267781 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.267898 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:15.271420 (+ 3522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:15.271426 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:15.271429 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:15.271462 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:15.274149 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:15.274974 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:15.274977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:15.274980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:15.275120 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:15.275546 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:15.275551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:15.275554 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:15.275887 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:15.276024 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:15.276714 (+ 690us) server_negotiation.cc:299] Negotiation successful
0810 06:42:15.276924 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":210,"threads_started":1}
I20260810 06:42:15.278162 28820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:15.260020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:41553 (local address 127.31.59.194:33205)
0810 06:42:15.260808 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:15.260854 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:15.262685 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:15.263394 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:15.263405 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:15.263916 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:42:15.264470 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:15.264487 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.265819 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.265823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:15.267097 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:15.267114 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.267956 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.267961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:15.268157 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:15.268763 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:42:15.268785 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:42:15.271197 (+ 2412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:42:15.274275 (+ 3078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:42:15.274283 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:42:15.274318 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:42:15.274868 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:42:15.275255 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:42:15.275259 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:42:15.275261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:42:15.275438 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:42:15.276000 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:42:15.276006 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:42:15.276354 (+ 348us) client_negotiation.cc:769] Sending connection context
0810 06:42:15.276584 (+ 230us) client_negotiation.cc:240] Negotiation successful
0810 06:42:15.276964 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":310,"threads_started":1}
I20260810 06:42:15.279524 28816 heartbeater.cc:347] Connected to a master server at 127.31.59.254:41553
I20260810 06:42:15.279810 28816 heartbeater.cc:464] Registering TS with master...
I20260810 06:42:15.280507 28816 heartbeater.cc:511] Master 127.31.59.254:41553 requested a full tablet report, sending...
I20260810 06:42:15.281903 28292 ts_manager.cc:194] Registered new tserver with Master: f9b2817ec71e4b4dbde231bcdb7fa506 (127.31.59.194:33705)
I20260810 06:42:15.282728 28292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:33205
WARNING: no policy specified for kudu/127.31.59.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
I20260810 06:42:15.320590 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:41553
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:36779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:15.644250 28828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:15.644519 28828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:15.644635 28828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:15.651525 28828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:15.651619 28828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:15.651762 28828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:42:15.660957 28828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:36779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:41553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:15.662142 28828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:15.663368 28828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:15.675750 28834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:15.675666 28836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:15.675715 28840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:15.676522 28828 server_base.cc:1034] running on GCE node
I20260810 06:42:15.677095 28828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:15.678095 28828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:15.679306 28828 hybrid_clock.cc:648] HybridClock initialized: now 1786344135679235 us; error 57 us; skew 500 ppm
I20260810 06:42:15.679606 28828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:15 dist-test-slave-4tdj krb5kdc[28213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344135, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:15.682468 28828 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:42:15.683848 28828 webserver.cc:460] Webserver started at http://127.31.59.195:32799/ using document root <none> and password file <none>
I20260810 06:42:15.684453 28828 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:15.684578 28828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:15.684857 28828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:15.687731 28828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "8551298dc4f74af2844a4210e7dabeff"
format_stamp: "Formatted at 2026-08-10 06:42:15 on dist-test-slave-4tdj"
server_key: "0d0b0def2627acad70563dcfb86bcfac"
server_key_iv: "5ee1f8e81e6f64c74025d58d7651d21e"
server_key_version: "encryptionkey@0"
I20260810 06:42:15.688436 28828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "8551298dc4f74af2844a4210e7dabeff"
format_stamp: "Formatted at 2026-08-10 06:42:15 on dist-test-slave-4tdj"
server_key: "0d0b0def2627acad70563dcfb86bcfac"
server_key_iv: "5ee1f8e81e6f64c74025d58d7651d21e"
server_key_version: "encryptionkey@0"
I20260810 06:42:15.693337 28828 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:42:15.696753 28852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:15.697966 28828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:42:15.698117 28828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "8551298dc4f74af2844a4210e7dabeff"
format_stamp: "Formatted at 2026-08-10 06:42:15 on dist-test-slave-4tdj"
server_key: "0d0b0def2627acad70563dcfb86bcfac"
server_key_iv: "5ee1f8e81e6f64c74025d58d7651d21e"
server_key_version: "encryptionkey@0"
I20260810 06:42:15.698287 28828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:15.717186 28828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:15.720785 28828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:15.721041 28828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:15.721975 28828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:42:15.723280 28828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:42:15.723361 28828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:15.723484 28828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:42:15.723546 28828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:15.753371 28828 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:36223
I20260810 06:42:15.753413 29016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:36223 every 8 connection(s)
I20260810 06:42:15.754594 28828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:42:15.755545 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 28828
I20260810 06:42:15.755718 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:42:15.756230 31983 external_mini_cluster.cc:1428] Setting key 272127c50c0d86875a7c17e59241e586
I20260810 06:42:15.759896 28600 heartbeater.cc:503] Master 127.31.59.254:41553 was elected leader, sending a full tablet report...
Aug 10 06:42:15 dist-test-slave-4tdj krb5kdc[28213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344135, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:42:15.774070 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:15.757224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:46403 (local address 127.31.59.254:41553)
0810 06:42:15.757418 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:15.757424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:15.759422 (+ 1998us) server_negotiation.cc:407] Connection header received
0810 06:42:15.761261 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:15.761266 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:15.761347 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:15.761451 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:15.762922 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.763694 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.765130 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.765520 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.765611 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:15.768744 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:15.768748 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:15.768752 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:15.768782 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:15.771611 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:15.772234 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:15.772239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:15.772242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:15.772364 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:15.772741 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:15.772743 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:15.772745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:15.772988 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:15.773105 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:15.773679 (+ 574us) server_negotiation.cc:299] Negotiation successful
0810 06:42:15.773836 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:42:15.775205 29020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:15.757792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:41553 (local address 127.31.59.195:46403)
0810 06:42:15.759301 (+ 1509us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:15.759338 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:15.761016 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:15.761665 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:15.761673 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:15.762167 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:42:15.762768 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:15.762786 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.763833 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.763836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:15.765020 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:15.765032 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.765702 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.765707 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:15.765890 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:15.766477 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:42:15.766503 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:42:15.768560 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:42:15.771717 (+ 3157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:42:15.771724 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:42:15.771734 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:42:15.772120 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:42:15.772459 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:42:15.772462 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:42:15.772464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:42:15.772659 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:42:15.773083 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:42:15.773089 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:42:15.773357 (+ 268us) client_negotiation.cc:769] Sending connection context
0810 06:42:15.773587 (+ 230us) client_negotiation.cc:240] Negotiation successful
0810 06:42:15.773919 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":334,"threads_started":1}
I20260810 06:42:15.776546 29017 heartbeater.cc:347] Connected to a master server at 127.31.59.254:41553
I20260810 06:42:15.776798 29017 heartbeater.cc:464] Registering TS with master...
I20260810 06:42:15.777448 29017 heartbeater.cc:511] Master 127.31.59.254:41553 requested a full tablet report, sending...
I20260810 06:42:15.778805 28292 ts_manager.cc:194] Registered new tserver with Master: 8551298dc4f74af2844a4210e7dabeff (127.31.59.195:36223)
I20260810 06:42:15.779582 28292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:46403
I20260810 06:42:15.783560 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:42:15.805877 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:15.787936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.31.59.254:41553)
0810 06:42:15.788123 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:15.788129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:15.788560 (+ 431us) server_negotiation.cc:407] Connection header received
0810 06:42:15.788694 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:15.788699 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:15.788770 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:15.788875 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:15.793739 (+ 4864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.794900 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.796884 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.797529 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.797649 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:15.799932 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:15.799939 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:15.799944 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:15.799982 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:15.803288 (+ 3306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:15.803923 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:15.803928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:15.803932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:15.804085 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:15.804388 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:15.804393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:15.804397 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:15.804784 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:15.805041 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:15.805300 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 06:42:15.805511 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:42:15.810077 28292 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:42:15.820832 28292 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:42:15.859583 29040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:15.834024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:33705 (local address 127.0.0.1:48030)
0810 06:42:15.849370 (+ 15346us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:15.849393 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:15.849508 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:15.850799 (+ 1291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:15.850807 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:15.850837 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:15.851257 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:15.851273 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.853281 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.853286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:15.855908 (+ 2622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:15.855922 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.858461 (+ 2539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.858467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:15.858760 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:15.858826 (+ 66us) client_negotiation.cc:769] Sending connection context
0810 06:42:15.858943 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 06:42:15.859159 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15229,"mutex_wait_us":116,"thread_start_us":18176,"threads_started":1}
I20260810 06:42:15.859823 29038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:15.834261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50284 (local address 127.31.59.195:36223)
0810 06:42:15.848574 (+ 14313us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:15.848585 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:15.850154 (+ 1569us) server_negotiation.cc:407] Connection header received
0810 06:42:15.850244 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:15.850251 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:15.850447 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:15.850620 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:15.851658 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.853946 (+ 2288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.856349 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.857864 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.857994 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:15.858237 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:15.859272 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0810 06:42:15.859436 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14080,"thread_start_us":13878,"threads_started":1}
I20260810 06:42:15.860390 29035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:15.833945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:36223 (local address 127.0.0.1:50284)
0810 06:42:15.849140 (+ 15195us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:15.849203 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:15.849421 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:15.850939 (+ 1518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:15.850944 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:15.850964 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:15.851442 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:15.851453 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.854108 (+ 2655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.854113 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:15.856176 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:15.856188 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.858420 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.858427 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:15.858734 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:15.858826 (+ 92us) client_negotiation.cc:769] Sending connection context
0810 06:42:15.858943 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 06:42:15.859159 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14898,"mutex_wait_us":68,"thread_start_us":15635,"threads_started":1}
I20260810 06:42:15.861223 29041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:15.836400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38690 (local address 127.31.59.193:34309)
0810 06:42:15.848934 (+ 12534us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:15.848943 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:15.852561 (+ 3618us) server_negotiation.cc:407] Connection header received
0810 06:42:15.852692 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:15.852701 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:15.852918 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:15.853090 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:15.853940 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.855401 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.857869 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.859352 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.859458 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:15.859670 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:15.860738 (+ 1068us) server_negotiation.cc:299] Negotiation successful
0810 06:42:15.860906 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12363,"thread_start_us":12223,"threads_started":1}
I20260810 06:42:15.861855 29037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:15.834202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48030 (local address 127.31.59.194:33705)
0810 06:42:15.849940 (+ 15738us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:15.849948 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:15.849977 (+ 29us) server_negotiation.cc:407] Connection header received
0810 06:42:15.850086 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:15.850093 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:15.850364 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:15.851480 (+ 1116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:15.851727 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.853163 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.856832 (+ 3669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.858225 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.861141 (+ 2916us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:15.861328 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:15.861394 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 06:42:15.861554 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15589,"thread_start_us":16307,"threads_started":1}
I20260810 06:42:15.862164 29036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:15.835887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34309 (local address 127.0.0.1:38690)
0810 06:42:15.852439 (+ 16552us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:15.852453 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:15.852557 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:15.853319 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:15.853323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:15.853351 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:15.853774 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:15.853789 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.855573 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.855578 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:15.857678 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:15.857695 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:15.860190 (+ 2495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:15.860197 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:15.860388 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:15.860421 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:42:15.860664 (+ 243us) client_negotiation.cc:240] Negotiation successful
0810 06:42:15.860798 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16440,"thread_start_us":13228,"threads_started":1}
I20260810 06:42:15.865567 28714 tablet_service.cc:1467] Processing CreateTablet for tablet 3dd0830da37d4a45916898eb97a23651 (DEFAULT_TABLE table=test-table [id=5537f61ae6a04ed79952f084c942cab2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:42:15.865576 28924 tablet_service.cc:1467] Processing CreateTablet for tablet 3dd0830da37d4a45916898eb97a23651 (DEFAULT_TABLE table=test-table [id=5537f61ae6a04ed79952f084c942cab2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:42:15.867094 28714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dd0830da37d4a45916898eb97a23651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:15.867089 28924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dd0830da37d4a45916898eb97a23651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:15.868556 28491 tablet_service.cc:1467] Processing CreateTablet for tablet 3dd0830da37d4a45916898eb97a23651 (DEFAULT_TABLE table=test-table [id=5537f61ae6a04ed79952f084c942cab2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:42:15.869917 28491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dd0830da37d4a45916898eb97a23651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:15.887912 29045 tablet_bootstrap.cc:492] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff: Bootstrap starting.
I20260810 06:42:15.890949 29044 tablet_bootstrap.cc:492] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506: Bootstrap starting.
I20260810 06:42:15.893635 29045 tablet_bootstrap.cc:654] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:15.894892 29045 log.cc:824] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:15.897501 29044 tablet_bootstrap.cc:654] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:15.898892 29044 log.cc:824] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:15.899783 29048 tablet_bootstrap.cc:492] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225: Bootstrap starting.
I20260810 06:42:15.903456 29048 tablet_bootstrap.cc:654] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:15.904604 29045 tablet_bootstrap.cc:492] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff: No bootstrap required, opened a new log
I20260810 06:42:15.904793 29048 log.cc:824] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:15.904901 29045 ts_tablet_manager.cc:1397] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.004s
I20260810 06:42:15.917448 29048 tablet_bootstrap.cc:492] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225: No bootstrap required, opened a new log
I20260810 06:42:15.917470 29044 tablet_bootstrap.cc:492] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506: No bootstrap required, opened a new log
I20260810 06:42:15.917773 29048 ts_tablet_manager.cc:1397] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.004s
I20260810 06:42:15.917943 29044 ts_tablet_manager.cc:1397] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506: Time spent bootstrapping tablet: real 0.027s user 0.006s sys 0.015s
I20260810 06:42:15.917655 29045 raft_consensus.cc:348] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8551298dc4f74af2844a4210e7dabeff" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 36223 } } peers { permanent_uuid: "f9b2817ec71e4b4dbde231bcdb7fa506" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 33705 } } peers { permanent_uuid: "606e53a085bb4d34af1ae6981f403225" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34309 } }
I20260810 06:42:15.918089 29045 raft_consensus.cc:374] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:15.918205 29045 raft_consensus.cc:729] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8551298dc4f74af2844a4210e7dabeff, State: Initialized, Role: FOLLOWER
I20260810 06:42:15.919039 29045 consensus_queue.cc:260] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8551298dc4f74af2844a4210e7dabeff" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 36223 } } peers { permanent_uuid: "f9b2817ec71e4b4dbde231bcdb7fa506" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 33705 } } peers { permanent_uuid: "606e53a085bb4d34af1ae6981f403225" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34309 } }
I20260810 06:42:15.927145 29045 ts_tablet_manager.cc:1428] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff: Time spent starting tablet: real 0.022s user 0.022s sys 0.000s
I20260810 06:42:15.927515 29017 heartbeater.cc:503] Master 127.31.59.254:41553 was elected leader, sending a full tablet report...
I20260810 06:42:15.928941 29044 raft_consensus.cc:348] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8551298dc4f74af2844a4210e7dabeff" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 36223 } } peers { permanent_uuid: "f9b2817ec71e4b4dbde231bcdb7fa506" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 33705 } } peers { permanent_uuid: "606e53a085bb4d34af1ae6981f403225" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34309 } }
I20260810 06:42:15.929464 29044 raft_consensus.cc:374] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:15.929605 29044 raft_consensus.cc:729] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9b2817ec71e4b4dbde231bcdb7fa506, State: Initialized, Role: FOLLOWER
I20260810 06:42:15.930215 29048 raft_consensus.cc:348] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8551298dc4f74af2844a4210e7dabeff" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 36223 } } peers { permanent_uuid: "f9b2817ec71e4b4dbde231bcdb7fa506" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 33705 } } peers { permanent_uuid: "606e53a085bb4d34af1ae6981f403225" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34309 } }
I20260810 06:42:15.930406 29044 consensus_queue.cc:260] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8551298dc4f74af2844a4210e7dabeff" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 36223 } } peers { permanent_uuid: "f9b2817ec71e4b4dbde231bcdb7fa506" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 33705 } } peers { permanent_uuid: "606e53a085bb4d34af1ae6981f403225" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34309 } }
I20260810 06:42:15.930673 29048 raft_consensus.cc:374] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:15.930761 29048 raft_consensus.cc:729] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606e53a085bb4d34af1ae6981f403225, State: Initialized, Role: FOLLOWER
I20260810 06:42:15.931448 29048 consensus_queue.cc:260] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8551298dc4f74af2844a4210e7dabeff" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 36223 } } peers { permanent_uuid: "f9b2817ec71e4b4dbde231bcdb7fa506" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 33705 } } peers { permanent_uuid: "606e53a085bb4d34af1ae6981f403225" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34309 } }
I20260810 06:42:15.934945 28816 heartbeater.cc:503] Master 127.31.59.254:41553 was elected leader, sending a full tablet report...
I20260810 06:42:15.936688 29044 ts_tablet_manager.cc:1428] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506: Time spent starting tablet: real 0.018s user 0.016s sys 0.000s
I20260810 06:42:15.943707 29048 ts_tablet_manager.cc:1428] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225: Time spent starting tablet: real 0.026s user 0.017s sys 0.008s
W20260810 06:42:15.982893 28601 tablet.cc:2369] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:42:16.006182 29018 tablet.cc:2369] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:42:16.008801 28817 tablet.cc:2369] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:42:16.153076 29055 raft_consensus.cc:482] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:42:16.153381 29055 raft_consensus.cc:504] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8551298dc4f74af2844a4210e7dabeff" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 36223 } } peers { permanent_uuid: "f9b2817ec71e4b4dbde231bcdb7fa506" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 33705 } } peers { permanent_uuid: "606e53a085bb4d34af1ae6981f403225" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34309 } }
I20260810 06:42:16.155234 29055 leader_election.cc:288] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8551298dc4f74af2844a4210e7dabeff (127.31.59.195:36223), 606e53a085bb4d34af1ae6981f403225 (127.31.59.193:34309)
I20260810 06:42:16.161775 29041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:16.155869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:48591 (local address 127.31.59.193:34309)
0810 06:42:16.156071 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:16.156077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:16.156377 (+ 300us) server_negotiation.cc:407] Connection header received
0810 06:42:16.156594 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:16.156599 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:16.156680 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:16.156766 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:16.157411 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:16.158185 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:16.160046 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:16.160830 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:16.160974 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:16.161058 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:16.161423 (+ 365us) server_negotiation.cc:299] Negotiation successful
0810 06:42:16.161541 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:42:16.161780 29038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:16.155670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:53953 (local address 127.31.59.195:36223)
0810 06:42:16.155877 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:16.155884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:16.156630 (+ 746us) server_negotiation.cc:407] Connection header received
0810 06:42:16.156780 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:16.156785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:16.156859 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:16.156951 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:16.157579 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:16.158330 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:16.160064 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:16.160826 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:16.160968 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:16.161042 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:16.161410 (+ 368us) server_negotiation.cc:299] Negotiation successful
0810 06:42:16.161545 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:42:16.161947 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:16.155528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:36223 (local address 127.31.59.194:53953)
0810 06:42:16.156522 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:16.156538 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:16.156661 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:16.156979 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:16.156982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:16.157007 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:16.157399 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:16.157410 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:16.158468 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:16.158472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:16.159921 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:16.159934 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:16.161062 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:16.161070 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:16.161244 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:16.161280 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:42:16.161351 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 06:42:16.161470 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"mutex_wait_us":427,"thread_start_us":270,"threads_started":1}
I20260810 06:42:16.162127 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:16.155520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34309 (local address 127.31.59.194:48591)
0810 06:42:16.156243 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:16.156266 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:16.156436 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:16.156787 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:16.156792 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:16.156818 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:16.157255 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:16.157266 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:16.158334 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:16.158338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:16.159885 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:16.159897 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:16.161102 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:16.161107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:16.161276 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:16.161299 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 06:42:16.161382 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 06:42:16.161503 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":8,"thread_start_us":329,"threads_started":1}
I20260810 06:42:16.163200 28952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dd0830da37d4a45916898eb97a23651" candidate_uuid: "f9b2817ec71e4b4dbde231bcdb7fa506" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8551298dc4f74af2844a4210e7dabeff" is_pre_election: true
I20260810 06:42:16.163200 28524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dd0830da37d4a45916898eb97a23651" candidate_uuid: "f9b2817ec71e4b4dbde231bcdb7fa506" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606e53a085bb4d34af1ae6981f403225" is_pre_election: true
I20260810 06:42:16.163758 28952 raft_consensus.cc:2454] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9b2817ec71e4b4dbde231bcdb7fa506 in term 0.
I20260810 06:42:16.163758 28524 raft_consensus.cc:2454] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9b2817ec71e4b4dbde231bcdb7fa506 in term 0.
I20260810 06:42:16.164433 28641 leader_election.cc:302] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606e53a085bb4d34af1ae6981f403225, f9b2817ec71e4b4dbde231bcdb7fa506; no voters:
I20260810 06:42:16.164800 29055 raft_consensus.cc:2781] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:42:16.164938 29055 raft_consensus.cc:482] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:42:16.165036 29055 raft_consensus.cc:3037] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:16.167289 29055 raft_consensus.cc:504] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8551298dc4f74af2844a4210e7dabeff" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 36223 } } peers { permanent_uuid: "f9b2817ec71e4b4dbde231bcdb7fa506" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 33705 } } peers { permanent_uuid: "606e53a085bb4d34af1ae6981f403225" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34309 } }
I20260810 06:42:16.167872 29055 leader_election.cc:288] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [CANDIDATE]: Term 1 election: Requested vote from peers 8551298dc4f74af2844a4210e7dabeff (127.31.59.195:36223), 606e53a085bb4d34af1ae6981f403225 (127.31.59.193:34309)
I20260810 06:42:16.168262 28952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dd0830da37d4a45916898eb97a23651" candidate_uuid: "f9b2817ec71e4b4dbde231bcdb7fa506" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8551298dc4f74af2844a4210e7dabeff"
I20260810 06:42:16.168340 28524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dd0830da37d4a45916898eb97a23651" candidate_uuid: "f9b2817ec71e4b4dbde231bcdb7fa506" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606e53a085bb4d34af1ae6981f403225"
I20260810 06:42:16.168511 28524 raft_consensus.cc:3037] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:16.168479 28952 raft_consensus.cc:3037] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:16.171650 28952 raft_consensus.cc:2454] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9b2817ec71e4b4dbde231bcdb7fa506 in term 1.
I20260810 06:42:16.171667 28524 raft_consensus.cc:2454] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9b2817ec71e4b4dbde231bcdb7fa506 in term 1.
I20260810 06:42:16.172037 28636 leader_election.cc:302] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8551298dc4f74af2844a4210e7dabeff, f9b2817ec71e4b4dbde231bcdb7fa506; no voters:
I20260810 06:42:16.172349 29055 raft_consensus.cc:2781] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:16.172773 29055 raft_consensus.cc:686] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [term 1 LEADER]: Becoming Leader. State: Replica: f9b2817ec71e4b4dbde231bcdb7fa506, State: Running, Role: LEADER
I20260810 06:42:16.173216 29055 consensus_queue.cc:237] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8551298dc4f74af2844a4210e7dabeff" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 36223 } } peers { permanent_uuid: "f9b2817ec71e4b4dbde231bcdb7fa506" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 33705 } } peers { permanent_uuid: "606e53a085bb4d34af1ae6981f403225" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34309 } }
I20260810 06:42:16.177591 28292 catalog_manager.cc:5495] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9b2817ec71e4b4dbde231bcdb7fa506 (127.31.59.194). New cstate: current_term: 1 leader_uuid: "f9b2817ec71e4b4dbde231bcdb7fa506" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8551298dc4f74af2844a4210e7dabeff" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 36223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9b2817ec71e4b4dbde231bcdb7fa506" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "606e53a085bb4d34af1ae6981f403225" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34309 } health_report { overall_health: UNKNOWN } } }
I20260810 06:42:16.223332 29037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:16.218126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48038 (local address 127.31.59.194:33705)
0810 06:42:16.218362 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:16.218367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:16.218388 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:42:16.218493 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:16.218497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:16.218573 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:16.218698 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:42:16.219078 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:16.219818 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:16.221231 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:16.221640 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:16.221749 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:16.222235 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:42:16.222744 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:42:16.222876 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:16.222981 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 06:42:16.223059 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:42:16.231324 28952 raft_consensus.cc:1261] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff [term 1 FOLLOWER]: Refusing update from remote peer f9b2817ec71e4b4dbde231bcdb7fa506: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:42:16.231325 28524 raft_consensus.cc:1261] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225 [term 1 FOLLOWER]: Refusing update from remote peer f9b2817ec71e4b4dbde231bcdb7fa506: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:42:16.232363 29055 consensus_queue.cc:1035] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [LEADER]: Connected to new peer: Peer: permanent_uuid: "606e53a085bb4d34af1ae6981f403225" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 34309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:42:16.232656 29062 consensus_queue.cc:1035] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8551298dc4f74af2844a4210e7dabeff" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 36223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:42:16.243700 29070 mvcc.cc:205] Tried to move back new op lower bound from 7316865581993496576 to 7316865581769043968. Current Snapshot: MvccSnapshot[applied={T|T < 7316865581993496576}]
I20260810 06:42:16.255138 28292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:42:16.255399 28292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:42:16.259402 28292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0903377353424db2ad1c67b6d26dc04e: Sending DeleteTablet for 3 replicas of tablet 3dd0830da37d4a45916898eb97a23651
I20260810 06:42:16.260519 28924 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dd0830da37d4a45916898eb97a23651 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:42:16 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:50284
I20260810 06:42:16.260583 28491 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dd0830da37d4a45916898eb97a23651 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:42:16 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:38690
I20260810 06:42:16.260809 28714 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dd0830da37d4a45916898eb97a23651 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:42:16 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:48030
I20260810 06:42:16.262336 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 28387
I20260810 06:42:16.275496 29089 tablet_replica.cc:330] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506: stopping tablet replica
I20260810 06:42:16.275993 29089 raft_consensus.cc:2229] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:42:16.277248 29084 tablet_replica.cc:330] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff: stopping tablet replica
I20260810 06:42:16.277643 29084 raft_consensus.cc:2229] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:42:16.277769 29089 raft_consensus.cc:2258] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:42:16.278199 29084 raft_consensus.cc:2258] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:42:16.280066 29084 ts_tablet_manager.cc:1905] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:42:16.280071 29089 ts_tablet_manager.cc:1905] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:42:16.286921 29084 ts_tablet_manager.cc:1918] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:42:16.287077 29084 log.cc:1196] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/3dd0830da37d4a45916898eb97a23651
I20260810 06:42:16.287569 29084 ts_tablet_manager.cc:1939] T 3dd0830da37d4a45916898eb97a23651 P 8551298dc4f74af2844a4210e7dabeff: Deleting consensus metadata
I20260810 06:42:16.289402 28270 catalog_manager.cc:4841] TS 8551298dc4f74af2844a4210e7dabeff (127.31.59.195:36223): tablet 3dd0830da37d4a45916898eb97a23651 (table test-table [id=5537f61ae6a04ed79952f084c942cab2]) successfully deleted
I20260810 06:42:16.290813 29085 tablet_replica.cc:330] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225: stopping tablet replica
I20260810 06:42:16.291234 29085 raft_consensus.cc:2229] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:42:16.291775 29085 raft_consensus.cc:2258] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:42:16.292611 29089 ts_tablet_manager.cc:1918] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:42:16.292834 29089 log.cc:1196] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/3dd0830da37d4a45916898eb97a23651
I20260810 06:42:16.293427 29089 ts_tablet_manager.cc:1939] T 3dd0830da37d4a45916898eb97a23651 P f9b2817ec71e4b4dbde231bcdb7fa506: Deleting consensus metadata
Aug 10 06:42:16 dist-test-slave-4tdj krb5kdc[28213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:42:16.294080 29085 ts_tablet_manager.cc:1905] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:42:16.295437 28271 catalog_manager.cc:4841] TS f9b2817ec71e4b4dbde231bcdb7fa506 (127.31.59.194:33705): tablet 3dd0830da37d4a45916898eb97a23651 (table test-table [id=5537f61ae6a04ed79952f084c942cab2]) successfully deleted
I20260810 06:42:16.300865 29041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:16.263510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38696 (local address 127.31.59.193:34309)
0810 06:42:16.263672 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:16.263678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:16.274729 (+ 11051us) server_negotiation.cc:407] Connection header received
0810 06:42:16.274966 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:16.274972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:16.275070 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:16.275171 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:16.277121 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:16.278237 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:16.288200 (+ 9963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:16.288832 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:16.288920 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:16.294759 (+ 5839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:16.294773 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:16.294789 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:16.294830 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:16.298117 (+ 3287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:16.298809 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:16.298816 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:16.298818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:16.298971 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:16.299343 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:16.299347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:16.299349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:16.299717 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:16.300024 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:16.300299 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 06:42:16.300525 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 06:42:16.301503 28593 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:16.390249 29085 ts_tablet_manager.cc:1918] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:42:16.390559 29085 log.cc:1196] T 3dd0830da37d4a45916898eb97a23651 P 606e53a085bb4d34af1ae6981f403225: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/3dd0830da37d4a45916898eb97a23651
I20260810 06:42:16.390885 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 28387
I20260810 06:42:16.417456 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 28606
W20260810 06:42:16.417412 28272 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.59.193:34309 (error 108)
I20260810 06:42:16.421298 29093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:16.418977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34309 (local address 127.0.0.1:38698)
0810 06:42:16.421026 (+ 2049us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:16.421134 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:34309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1930,"thread_start_us":1838,"threads_started":1}
W20260810 06:42:16.421496 28272 catalog_manager.cc:4584] TS 606e53a085bb4d34af1ae6981f403225 (127.31.59.193:34309): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3dd0830da37d4a45916898eb97a23651: Network error: Client connection negotiation failed: client connection to 127.31.59.193:34309: connect: Connection refused (error 111)
Aug 10 06:42:16 dist-test-slave-4tdj krb5kdc[28213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.194@KRBTEST.COM
I20260810 06:42:16.435181 29037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:16.418419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48044 (local address 127.31.59.194:33705)
0810 06:42:16.418605 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:16.418611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:16.418631 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:42:16.418772 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:16.418777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:16.418866 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:16.418974 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:16.420755 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:16.422023 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:16.423485 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:16.424167 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:16.424306 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:16.428294 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:16.428311 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:16.428331 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:16.428383 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:16.432184 (+ 3801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:16.432879 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:16.432889 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:16.432893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:16.433068 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:16.433460 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:16.433466 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:16.433469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:16.433911 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:16.434271 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:16.434564 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 06:42:16.434785 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:42:16.435899 28810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:16.474970 29093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:16.474481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34309 (local address 127.0.0.1:38702)
0810 06:42:16.474693 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:16.474830 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:34309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 06:42:16.476042 29093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:16.475622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34309 (local address 127.0.0.1:38708)
0810 06:42:16.475805 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:16.475891 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:34309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 06:42:16.536778 29093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:16.536271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34309 (local address 127.0.0.1:38722)
0810 06:42:16.536516 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:16.536621 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:34309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260810 06:42:16.537837 29093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:16.537495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34309 (local address 127.0.0.1:38730)
0810 06:42:16.537630 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:16.537703 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:34309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 06:42:16.548750 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 28606
I20260810 06:42:16.572853 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 28828
Aug 10 06:42:16 dist-test-slave-4tdj krb5kdc[28213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.195@KRBTEST.COM
I20260810 06:42:16.587476 29038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:16.573576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50292 (local address 127.31.59.195:36223)
0810 06:42:16.573816 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:16.573822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:16.573903 (+ 81us) server_negotiation.cc:407] Connection header received
0810 06:42:16.574106 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:16.574111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:16.574196 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:16.574280 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:16.575660 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:16.576484 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:16.577704 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:16.578098 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:16.578208 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:16.582390 (+ 4182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:16.582403 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:16.582420 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:16.582453 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:16.584877 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:16.585494 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:16.585502 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:16.585504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:16.585635 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:16.585970 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:16.585974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:16.585976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:16.586378 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:16.586685 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:16.586941 (+ 256us) server_negotiation.cc:299] Negotiation successful
0810 06:42:16.587149 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260810 06:42:16.588100 29011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:16.618788 29093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:16.618329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34309 (local address 127.0.0.1:38732)
0810 06:42:16.618510 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:16.618626 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:34309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 06:42:16.619825 29093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:16.619420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34309 (local address 127.0.0.1:38740)
0810 06:42:16.619611 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:16.619694 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:34309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260810 06:42:16.675779 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 28828
I20260810 06:42:16.699263 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 28240
I20260810 06:42:16.700493 28339 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:16.808629 29093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:16.808119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:34309 (local address 127.0.0.1:38742)
0810 06:42:16.808306 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:16.808439 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:34309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 06:42:16.808996 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 28240
2026-08-10T06:42:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4616 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T06:42:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:42:16Z Disabled control of system clock
I20260810 06:42:16.900391 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40063
--webserver_interface=127.31.59.254
--webserver_port=0
--builtin_ntp_servers=127.31.59.212:40277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:40063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:42:17.232957 29109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:17.233243 29109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:17.233371 29109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:17.240368 29109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:42:17.240475 29109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:17.240543 29109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:17.240635 29109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:42:17.240700 29109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:42:17.249594 29109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:40277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:40063
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:17.250842 29109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:17.252619 29109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:17.263594 29117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:17.263672 29122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:17.263638 29118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:17.264416 29109 server_base.cc:1034] running on GCE node
I20260810 06:42:17.265147 29109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:17.266487 29109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:17.267699 29109 hybrid_clock.cc:648] HybridClock initialized: now 1786344137267644 us; error 45 us; skew 500 ppm
I20260810 06:42:17.268014 29109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:42:17.274556 29109 webserver.cc:460] Webserver started at http://127.31.59.254:44341/ using document root <none> and password file <none>
I20260810 06:42:17.275195 29109 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:17.275298 29109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:17.275604 29109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:17.278458 29109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "f67b4d6e3ee147e690da6ed3cb7db7d0"
format_stamp: "Formatted at 2026-08-10 06:42:17 on dist-test-slave-4tdj"
server_key: "a5a98fecacc13f578b2eebca46bca02a"
server_key_iv: "bcb1dc32c748743cf06e8708a0cc942a"
server_key_version: "encryptionkey@0"
I20260810 06:42:17.279212 29109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "f67b4d6e3ee147e690da6ed3cb7db7d0"
format_stamp: "Formatted at 2026-08-10 06:42:17 on dist-test-slave-4tdj"
server_key: "a5a98fecacc13f578b2eebca46bca02a"
server_key_iv: "bcb1dc32c748743cf06e8708a0cc942a"
server_key_version: "encryptionkey@0"
I20260810 06:42:17.284163 29109 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260810 06:42:17.287585 29134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:17.288851 29109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:42:17.289011 29109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "f67b4d6e3ee147e690da6ed3cb7db7d0"
format_stamp: "Formatted at 2026-08-10 06:42:17 on dist-test-slave-4tdj"
server_key: "a5a98fecacc13f578b2eebca46bca02a"
server_key_iv: "bcb1dc32c748743cf06e8708a0cc942a"
server_key_version: "encryptionkey@0"
I20260810 06:42:17.289204 29109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:17.307730 29109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:17.308507 29109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:17.308773 29109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:17.326108 29109 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:40063
I20260810 06:42:17.326154 29210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:40063 every 8 connection(s)
I20260810 06:42:17.327401 29109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:42:17.331063 29211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:17.334604 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 29109
I20260810 06:42:17.334782 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:42:17.335337 31983 external_mini_cluster.cc:1428] Setting key 8f83a5c686eb157da104c1e06c968a00
I20260810 06:42:17.343071 29211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0: Bootstrap starting.
I20260810 06:42:17.346016 29211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:17.347128 29211 log.cc:824] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:17.348347 29217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:17.337208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39130 (local address 127.31.59.254:40063)
0810 06:42:17.338099 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:17.338109 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:17.338150 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:42:17.339784 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:17.339804 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:17.340266 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:17.340621 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:17.341185 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:17.342452 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:17.343899 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:17.344567 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:17.344734 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:17.345819 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:17.345826 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:17.345838 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:17.346416 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:17.346509 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:17.346530 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:17.346636 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:17.346844 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 06:42:17.347167 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":341,"threads_started":1}
I20260810 06:42:17.350931 29211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0: No bootstrap required, opened a new log
I20260810 06:42:17.357717 29211 raft_consensus.cc:348] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f67b4d6e3ee147e690da6ed3cb7db7d0" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40063 } }
I20260810 06:42:17.358070 29211 raft_consensus.cc:374] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:17.358171 29211 raft_consensus.cc:729] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f67b4d6e3ee147e690da6ed3cb7db7d0, State: Initialized, Role: FOLLOWER
I20260810 06:42:17.358719 29211 consensus_queue.cc:260] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f67b4d6e3ee147e690da6ed3cb7db7d0" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40063 } }
I20260810 06:42:17.358937 29211 raft_consensus.cc:388] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:42:17.359061 29211 raft_consensus.cc:482] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:42:17.359201 29211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:17.361191 29211 raft_consensus.cc:504] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f67b4d6e3ee147e690da6ed3cb7db7d0" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40063 } }
I20260810 06:42:17.361642 29211 leader_election.cc:302] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f67b4d6e3ee147e690da6ed3cb7db7d0; no voters:
I20260810 06:42:17.362183 29211 leader_election.cc:288] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:42:17.362327 29219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:17.362979 29219 raft_consensus.cc:686] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0 [term 1 LEADER]: Becoming Leader. State: Replica: f67b4d6e3ee147e690da6ed3cb7db7d0, State: Running, Role: LEADER
I20260810 06:42:17.363449 29219 consensus_queue.cc:237] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f67b4d6e3ee147e690da6ed3cb7db7d0" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40063 } }
I20260810 06:42:17.364008 29211 sys_catalog.cc:564] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:42:17.365958 29222 sys_catalog.cc:455] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f67b4d6e3ee147e690da6ed3cb7db7d0. Latest consensus state: current_term: 1 leader_uuid: "f67b4d6e3ee147e690da6ed3cb7db7d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f67b4d6e3ee147e690da6ed3cb7db7d0" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40063 } } }
I20260810 06:42:17.366267 29222 sys_catalog.cc:458] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:17.366992 29220 sys_catalog.cc:455] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f67b4d6e3ee147e690da6ed3cb7db7d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f67b4d6e3ee147e690da6ed3cb7db7d0" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40063 } } }
I20260810 06:42:17.367275 29220 sys_catalog.cc:458] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:17.367892 29225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:42:17.373358 29225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:42:17.388679 29225 catalog_manager.cc:1347] Generated new cluster ID: dcc06736c6614c8e9f694eb8eb498332
I20260810 06:42:17.388792 29225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:42:17.403772 29225 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:42:17.404949 29225 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:42:17.416918 29225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0: Generated new TSK 0
I20260810 06:42:17.417712 29225 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:42:17.429337 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40063
--builtin_ntp_servers=127.31.59.212:40277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:42:17.806159 29250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:17.806535 29250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:17.806643 29250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:17.813556 29250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:17.813644 29250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:17.813747 29250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:42:17.822414 29250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:40277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:17.823544 29250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:17.825263 29250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:17.837567 29255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:17.837620 29256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:17.837774 29260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:17.838455 29250 server_base.cc:1034] running on GCE node
I20260810 06:42:17.839044 29250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:17.839928 29250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:17.841131 29250 hybrid_clock.cc:648] HybridClock initialized: now 1786344137841091 us; error 39 us; skew 500 ppm
I20260810 06:42:17.841429 29250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:42:17.843596 29250 webserver.cc:460] Webserver started at http://127.31.59.193:37695/ using document root <none> and password file <none>
I20260810 06:42:17.844236 29250 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:17.844338 29250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:17.844643 29250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:17.847478 29250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "b7841ae2f1704cb19f592e470376ddbe"
format_stamp: "Formatted at 2026-08-10 06:42:17 on dist-test-slave-4tdj"
server_key: "7ef7c1758b4072255e30c820580eddf1"
server_key_iv: "9f5e94ef46ef5e0612b09bf41b616435"
server_key_version: "encryptionkey@0"
I20260810 06:42:17.848248 29250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "b7841ae2f1704cb19f592e470376ddbe"
format_stamp: "Formatted at 2026-08-10 06:42:17 on dist-test-slave-4tdj"
server_key: "7ef7c1758b4072255e30c820580eddf1"
server_key_iv: "9f5e94ef46ef5e0612b09bf41b616435"
server_key_version: "encryptionkey@0"
I20260810 06:42:17.853072 29250 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:42:17.856285 29269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:17.857462 29250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:42:17.857612 29250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "b7841ae2f1704cb19f592e470376ddbe"
format_stamp: "Formatted at 2026-08-10 06:42:17 on dist-test-slave-4tdj"
server_key: "7ef7c1758b4072255e30c820580eddf1"
server_key_iv: "9f5e94ef46ef5e0612b09bf41b616435"
server_key_version: "encryptionkey@0"
I20260810 06:42:17.857795 29250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:17.876132 29250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:17.876876 29250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:17.877153 29250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:17.878043 29250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:42:17.879489 29250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:42:17.879601 29250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:17.879702 29250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:42:17.879770 29250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:17.910784 29250 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:41267
I20260810 06:42:17.910849 29449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:41267 every 8 connection(s)
I20260810 06:42:17.912087 29250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:42:17.921579 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 29250
I20260810 06:42:17.921768 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:42:17.922212 31983 external_mini_cluster.cc:1428] Setting key 54ddeb5fa16a580f741ae20a7224f7db
I20260810 06:42:17.924606 29454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:17.914614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:53863 (local address 127.31.59.254:40063)
0810 06:42:17.915054 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:17.915059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:17.916241 (+ 1182us) server_negotiation.cc:407] Connection header received
0810 06:42:17.918070 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:17.918076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:17.918154 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:17.918250 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:17.919329 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:17.920029 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:17.921902 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:17.922360 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:17.922461 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:17.923853 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:17.923859 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:17.923864 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:17.923884 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:17.923933 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:17.923949 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:17.924042 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:17.924267 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 06:42:17.924342 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":222,"threads_started":1}
I20260810 06:42:17.925241 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40063
--builtin_ntp_servers=127.31.59.212:40277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:42:17.925932 29455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:17.915114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40063 (local address 127.31.59.193:53863)
0810 06:42:17.916097 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:17.916135 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:17.917813 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:17.918493 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:17.918501 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:17.918586 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:42:17.919166 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:17.919185 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:17.920174 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:17.920178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:17.921748 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:17.921765 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:17.922582 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:17.922590 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:17.922865 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:17.923560 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:42:17.923563 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:42:17.923599 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:42:17.923675 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:42:17.924051 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:42:17.924059 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:42:17.924065 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 06:42:17.924238 (+ 173us) client_negotiation.cc:240] Negotiation successful
0810 06:42:17.924598 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":262,"threads_started":1}
I20260810 06:42:17.927291 29451 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40063
I20260810 06:42:17.927585 29451 heartbeater.cc:464] Registering TS with master...
I20260810 06:42:17.928404 29451 heartbeater.cc:511] Master 127.31.59.254:40063 requested a full tablet report, sending...
I20260810 06:42:17.930562 29158 ts_manager.cc:194] Registered new tserver with Master: b7841ae2f1704cb19f592e470376ddbe (127.31.59.193:41267)
I20260810 06:42:17.932042 29158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.193:53863
W20260810 06:42:18.251941 29457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:18.252220 29457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:18.252359 29457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:18.260030 29457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:18.260129 29457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:18.260283 29457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:42:18.269666 29457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:40277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:18.270879 29457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:18.272676 29457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:18.285317 29466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:18.285399 29469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:18.285434 29467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:18.286226 29457 server_base.cc:1034] running on GCE node
I20260810 06:42:18.286875 29457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:18.287804 29457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:18.289021 29457 hybrid_clock.cc:648] HybridClock initialized: now 1786344138288975 us; error 32 us; skew 500 ppm
I20260810 06:42:18.289340 29457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:42:18.291594 29457 webserver.cc:460] Webserver started at http://127.31.59.194:46557/ using document root <none> and password file <none>
I20260810 06:42:18.292254 29457 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:18.292356 29457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:18.292676 29457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:18.295548 29457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "129c72857944432987f5f248f7735a86"
format_stamp: "Formatted at 2026-08-10 06:42:18 on dist-test-slave-4tdj"
server_key: "5dfb7a25006a6d5c802577785b25041c"
server_key_iv: "bb16bde19768f9c0f0cf2430d477fddf"
server_key_version: "encryptionkey@0"
I20260810 06:42:18.296356 29457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "129c72857944432987f5f248f7735a86"
format_stamp: "Formatted at 2026-08-10 06:42:18 on dist-test-slave-4tdj"
server_key: "5dfb7a25006a6d5c802577785b25041c"
server_key_iv: "bb16bde19768f9c0f0cf2430d477fddf"
server_key_version: "encryptionkey@0"
I20260810 06:42:18.301539 29457 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:42:18.305083 29480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:18.306396 29457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:42:18.306560 29457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "129c72857944432987f5f248f7735a86"
format_stamp: "Formatted at 2026-08-10 06:42:18 on dist-test-slave-4tdj"
server_key: "5dfb7a25006a6d5c802577785b25041c"
server_key_iv: "bb16bde19768f9c0f0cf2430d477fddf"
server_key_version: "encryptionkey@0"
I20260810 06:42:18.306764 29457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:18.324936 29457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:18.325768 29457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:18.326095 29457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:18.327137 29457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:42:18.328745 29457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:42:18.328855 29457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:18.328982 29457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:42:18.329061 29457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:18.361929 29457 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:41291
I20260810 06:42:18.361989 29669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:41291 every 8 connection(s)
I20260810 06:42:18.363365 29457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:42:18.372946 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 29457
I20260810 06:42:18.373140 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:42:18.373695 31983 external_mini_cluster.cc:1428] Setting key 77d1500f2a404776aa0f5d52710f2e36
I20260810 06:42:18.376266 29454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:18.366143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:40355 (local address 127.31.59.254:40063)
0810 06:42:18.366393 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:18.366401 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:18.367842 (+ 1441us) server_negotiation.cc:407] Connection header received
0810 06:42:18.369644 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:18.369648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:18.369718 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:18.369809 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:18.370886 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.371752 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.373565 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.373983 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.374086 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:18.375506 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:18.375512 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:18.375516 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:18.375538 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:18.375614 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:18.375632 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:18.375716 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:18.375970 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 06:42:18.376044 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:42:18.376808 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40063
--builtin_ntp_servers=127.31.59.212:40277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:42:18.377563 29674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:18.366764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40063 (local address 127.31.59.194:40355)
0810 06:42:18.367687 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:18.367732 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:18.369370 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:18.370018 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:18.370027 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:18.370111 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:42:18.370718 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:18.370735 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.371897 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.371901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:18.373415 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:18.373430 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.374207 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.374216 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:18.374489 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:18.375202 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:42:18.375205 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:42:18.375242 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:42:18.375318 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:42:18.375744 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:42:18.375755 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:42:18.375763 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:42:18.375954 (+ 191us) client_negotiation.cc:240] Negotiation successful
0810 06:42:18.376319 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":305,"threads_started":1}
I20260810 06:42:18.378829 29670 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40063
I20260810 06:42:18.379130 29670 heartbeater.cc:464] Registering TS with master...
I20260810 06:42:18.379959 29670 heartbeater.cc:511] Master 127.31.59.254:40063 requested a full tablet report, sending...
I20260810 06:42:18.381361 29158 ts_manager.cc:194] Registered new tserver with Master: 129c72857944432987f5f248f7735a86 (127.31.59.194:41291)
I20260810 06:42:18.382126 29158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.194:40355
W20260810 06:42:18.704666 29676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:18.704929 29676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:18.705027 29676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:18.712412 29676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:18.712518 29676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:18.712631 29676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:42:18.721478 29676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:40277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:18.722772 29676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:18.724465 29676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:18.736956 29683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:18.737015 29684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:18.736989 29687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:18.737527 29676 server_base.cc:1034] running on GCE node
I20260810 06:42:18.738215 29676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:18.739148 29676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:18.740394 29676 hybrid_clock.cc:648] HybridClock initialized: now 1786344138740329 us; error 48 us; skew 500 ppm
I20260810 06:42:18.740746 29676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:42:18.742959 29676 webserver.cc:460] Webserver started at http://127.31.59.195:33305/ using document root <none> and password file <none>
I20260810 06:42:18.743659 29676 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:18.743765 29676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:18.744071 29676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:18.747011 29676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "712d32df021447a2832043d7665e0c59"
format_stamp: "Formatted at 2026-08-10 06:42:18 on dist-test-slave-4tdj"
server_key: "c0500c018a52423dca201bca726c3aef"
server_key_iv: "0d438ab06b96bdbe0356f6754b62232e"
server_key_version: "encryptionkey@0"
I20260810 06:42:18.747787 29676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "712d32df021447a2832043d7665e0c59"
format_stamp: "Formatted at 2026-08-10 06:42:18 on dist-test-slave-4tdj"
server_key: "c0500c018a52423dca201bca726c3aef"
server_key_iv: "0d438ab06b96bdbe0356f6754b62232e"
server_key_version: "encryptionkey@0"
I20260810 06:42:18.753010 29676 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:42:18.756440 29698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:18.757727 29676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:42:18.757874 29676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "712d32df021447a2832043d7665e0c59"
format_stamp: "Formatted at 2026-08-10 06:42:18 on dist-test-slave-4tdj"
server_key: "c0500c018a52423dca201bca726c3aef"
server_key_iv: "0d438ab06b96bdbe0356f6754b62232e"
server_key_version: "encryptionkey@0"
I20260810 06:42:18.758070 29676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:18.781701 29676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:18.782567 29676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:18.783063 29676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:18.784138 29676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:42:18.785722 29676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:42:18.785840 29676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:18.785967 29676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:42:18.786058 29676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:18.818388 29676 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:37201
I20260810 06:42:18.818450 29889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:37201 every 8 connection(s)
I20260810 06:42:18.819705 29676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:42:18.825344 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 29676
I20260810 06:42:18.825541 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:42:18.825994 31983 external_mini_cluster.cc:1428] Setting key ea7a262ba0786817e00a31e0584610c5
I20260810 06:42:18.832058 29454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:18.822264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:41635 (local address 127.31.59.254:40063)
0810 06:42:18.822494 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:18.822498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:18.823887 (+ 1389us) server_negotiation.cc:407] Connection header received
0810 06:42:18.825691 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:18.825697 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:18.825775 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:18.825862 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:18.827079 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.828004 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.829570 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.829956 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.830058 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:18.831421 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:18.831426 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:18.831429 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:18.831444 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:18.831479 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:18.831493 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:18.831561 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:18.831739 (+ 178us) server_negotiation.cc:299] Negotiation successful
0810 06:42:18.831813 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260810 06:42:18.832989 29892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:18.822849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40063 (local address 127.31.59.195:41635)
0810 06:42:18.823747 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:18.823782 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:18.825416 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:18.826107 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:18.826119 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:18.826217 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:42:18.826918 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:18.826937 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.828151 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.828155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:18.829429 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:18.829442 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.830222 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.830228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:18.830495 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:18.831146 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:42:18.831149 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:42:18.831186 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:42:18.831263 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:42:18.831573 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:42:18.831579 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:42:18.831585 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 06:42:18.831716 (+ 131us) client_negotiation.cc:240] Negotiation successful
0810 06:42:18.831992 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":276,"threads_started":1}
I20260810 06:42:18.834331 29890 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40063
I20260810 06:42:18.834611 29890 heartbeater.cc:464] Registering TS with master...
I20260810 06:42:18.835299 29890 heartbeater.cc:511] Master 127.31.59.254:40063 requested a full tablet report, sending...
I20260810 06:42:18.836584 29158 ts_manager.cc:194] Registered new tserver with Master: 712d32df021447a2832043d7665e0c59 (127.31.59.195:37201)
I20260810 06:42:18.837354 29158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.195:41635
I20260810 06:42:18.840818 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:42:18.850430 29454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:18.844786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33578 (local address 127.31.59.254:40063)
0810 06:42:18.844929 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:18.844934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:18.845388 (+ 454us) server_negotiation.cc:407] Connection header received
0810 06:42:18.845560 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:18.845564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:18.845635 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:18.845715 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:18.846203 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.846929 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.848251 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.848684 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.848791 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:18.849773 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:18.849778 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:18.849781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:18.849797 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:18.849827 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:18.849839 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:18.849909 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:18.850039 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 06:42:18.850124 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:42:18.853713 29158 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:42:18.860448 29158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:42:18.884379 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:18.872155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:41291 (local address 127.0.0.1:60058)
0810 06:42:18.872913 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:18.872959 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:18.873127 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:18.875038 (+ 1911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:18.875045 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:18.875076 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:18.875493 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:18.875509 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.877677 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.877684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:18.880729 (+ 3045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:18.880747 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.882785 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.882793 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:18.883015 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:18.883078 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 06:42:18.883190 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 06:42:18.883374 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":196,"thread_start_us":345,"threads_started":1}
I20260810 06:42:18.885643 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:18.872401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60058 (local address 127.31.59.194:41291)
0810 06:42:18.874514 (+ 2113us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:18.874520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:18.874543 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:42:18.874627 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:18.874633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:18.874805 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:18.875722 (+ 917us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:18.876008 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.877508 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.880950 (+ 3442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.882543 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.884079 (+ 1536us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:18.884882 (+ 803us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:18.885080 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 06:42:18.885283 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1987,"thread_start_us":1887,"threads_started":1}
I20260810 06:42:18.886221 29907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:18.872327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:37201 (local address 127.0.0.1:60316)
0810 06:42:18.874842 (+ 2515us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:18.874859 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:18.874974 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:18.875940 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:18.875945 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:18.875971 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:18.876359 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:18.876369 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.878489 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.878495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:18.881489 (+ 2994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:18.881501 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.885372 (+ 3871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.885379 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:18.885585 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:18.885620 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 06:42:18.885698 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 06:42:18.885829 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2405,"mutex_wait_us":1,"thread_start_us":2277,"threads_started":1}
I20260810 06:42:18.887035 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:18.873446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60316 (local address 127.31.59.195:37201)
0810 06:42:18.873975 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:18.873983 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:18.875111 (+ 1128us) server_negotiation.cc:407] Connection header received
0810 06:42:18.875219 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:18.875227 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:18.875466 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:18.875649 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:18.876623 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.878315 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.881698 (+ 3383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.883352 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.883488 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:18.883779 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:18.886488 (+ 2709us) server_negotiation.cc:299] Negotiation successful
0810 06:42:18.886674 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":571,"threads_started":1}
I20260810 06:42:18.889885 29559 tablet_service.cc:1467] Processing CreateTablet for tablet b4daaef484214afbb5bad2b61f1f4705 (DEFAULT_TABLE table=test-table [id=48ec9bc7d28e43b8b86257dee1e2281c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:42:18.891456 29559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4daaef484214afbb5bad2b61f1f4705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:18.891252 29779 tablet_service.cc:1467] Processing CreateTablet for tablet b4daaef484214afbb5bad2b61f1f4705 (DEFAULT_TABLE table=test-table [id=48ec9bc7d28e43b8b86257dee1e2281c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:42:18.892709 29779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4daaef484214afbb5bad2b61f1f4705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:18.894124 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:18.874983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:41267 (local address 127.0.0.1:55960)
0810 06:42:18.885108 (+ 10125us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:18.885126 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:18.885235 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:18.888783 (+ 3548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:18.888788 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:18.888812 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:18.888962 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:18.888973 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.890569 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.890574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:18.891766 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:18.891779 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.893337 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.893343 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:18.893566 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:18.893590 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 06:42:18.893666 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 06:42:18.893804 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10013,"thread_start_us":12642,"threads_started":1}
I20260810 06:42:18.894230 29919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:18.875266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55960 (local address 127.31.59.193:41267)
0810 06:42:18.888223 (+ 12957us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:18.888231 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:18.888259 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:42:18.888349 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:18.888356 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:18.888605 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:18.888784 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:18.889149 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.890429 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.891968 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:18.893140 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:18.893259 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:18.893484 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:18.893725 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 06:42:18.893875 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12801,"thread_start_us":12688,"threads_started":1}
I20260810 06:42:18.898185 29345 tablet_service.cc:1467] Processing CreateTablet for tablet b4daaef484214afbb5bad2b61f1f4705 (DEFAULT_TABLE table=test-table [id=48ec9bc7d28e43b8b86257dee1e2281c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:42:18.899523 29345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4daaef484214afbb5bad2b61f1f4705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:18.905948 29923 tablet_bootstrap.cc:492] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86: Bootstrap starting.
I20260810 06:42:18.907744 29925 tablet_bootstrap.cc:492] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59: Bootstrap starting.
I20260810 06:42:18.909443 29923 tablet_bootstrap.cc:654] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:18.910974 29923 log.cc:824] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:18.912266 29925 tablet_bootstrap.cc:654] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:18.913477 29925 log.cc:824] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:18.916647 29929 tablet_bootstrap.cc:492] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe: Bootstrap starting.
I20260810 06:42:18.921713 29929 tablet_bootstrap.cc:654] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:18.924949 29925 tablet_bootstrap.cc:492] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59: No bootstrap required, opened a new log
I20260810 06:42:18.925251 29925 ts_tablet_manager.cc:1397] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.010s
I20260810 06:42:18.925267 29923 tablet_bootstrap.cc:492] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86: No bootstrap required, opened a new log
I20260810 06:42:18.925513 29923 ts_tablet_manager.cc:1397] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.000s
I20260810 06:42:18.925726 29929 log.cc:824] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:18.928521 29929 tablet_bootstrap.cc:492] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe: No bootstrap required, opened a new log
I20260810 06:42:18.928781 29929 ts_tablet_manager.cc:1397] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260810 06:42:18.933199 29923 raft_consensus.cc:348] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129c72857944432987f5f248f7735a86" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 41291 } } peers { permanent_uuid: "712d32df021447a2832043d7665e0c59" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37201 } } peers { permanent_uuid: "b7841ae2f1704cb19f592e470376ddbe" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 41267 } }
I20260810 06:42:18.933525 29923 raft_consensus.cc:374] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:18.933622 29923 raft_consensus.cc:729] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 129c72857944432987f5f248f7735a86, State: Initialized, Role: FOLLOWER
I20260810 06:42:18.934196 29923 consensus_queue.cc:260] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129c72857944432987f5f248f7735a86" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 41291 } } peers { permanent_uuid: "712d32df021447a2832043d7665e0c59" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37201 } } peers { permanent_uuid: "b7841ae2f1704cb19f592e470376ddbe" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 41267 } }
I20260810 06:42:18.935096 29451 heartbeater.cc:503] Master 127.31.59.254:40063 was elected leader, sending a full tablet report...
I20260810 06:42:18.935654 29670 heartbeater.cc:503] Master 127.31.59.254:40063 was elected leader, sending a full tablet report...
I20260810 06:42:18.935690 29925 raft_consensus.cc:348] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129c72857944432987f5f248f7735a86" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 41291 } } peers { permanent_uuid: "712d32df021447a2832043d7665e0c59" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37201 } } peers { permanent_uuid: "b7841ae2f1704cb19f592e470376ddbe" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 41267 } }
I20260810 06:42:18.936101 29925 raft_consensus.cc:374] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:18.936190 29925 raft_consensus.cc:729] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 712d32df021447a2832043d7665e0c59, State: Initialized, Role: FOLLOWER
I20260810 06:42:18.936246 29923 ts_tablet_manager.cc:1428] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260810 06:42:18.937070 29925 consensus_queue.cc:260] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129c72857944432987f5f248f7735a86" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 41291 } } peers { permanent_uuid: "712d32df021447a2832043d7665e0c59" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37201 } } peers { permanent_uuid: "b7841ae2f1704cb19f592e470376ddbe" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 41267 } }
I20260810 06:42:18.938529 29890 heartbeater.cc:503] Master 127.31.59.254:40063 was elected leader, sending a full tablet report...
I20260810 06:42:18.939127 29925 ts_tablet_manager.cc:1428] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59: Time spent starting tablet: real 0.014s user 0.008s sys 0.006s
I20260810 06:42:18.940371 29929 raft_consensus.cc:348] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129c72857944432987f5f248f7735a86" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 41291 } } peers { permanent_uuid: "712d32df021447a2832043d7665e0c59" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37201 } } peers { permanent_uuid: "b7841ae2f1704cb19f592e470376ddbe" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 41267 } }
I20260810 06:42:18.940816 29929 raft_consensus.cc:374] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:18.940905 29929 raft_consensus.cc:729] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7841ae2f1704cb19f592e470376ddbe, State: Initialized, Role: FOLLOWER
I20260810 06:42:18.941493 29929 consensus_queue.cc:260] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129c72857944432987f5f248f7735a86" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 41291 } } peers { permanent_uuid: "712d32df021447a2832043d7665e0c59" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37201 } } peers { permanent_uuid: "b7841ae2f1704cb19f592e470376ddbe" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 41267 } }
I20260810 06:42:18.943255 29929 ts_tablet_manager.cc:1428] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe: Time spent starting tablet: real 0.014s user 0.007s sys 0.007s
W20260810 06:42:19.071405 29891 tablet.cc:2369] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:42:19.115502 29673 tablet.cc:2369] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:42:19.164534 29452 tablet.cc:2369] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:42:19.236310 29932 raft_consensus.cc:482] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:42:19.236614 29932 raft_consensus.cc:504] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129c72857944432987f5f248f7735a86" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 41291 } } peers { permanent_uuid: "712d32df021447a2832043d7665e0c59" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37201 } } peers { permanent_uuid: "b7841ae2f1704cb19f592e470376ddbe" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 41267 } }
I20260810 06:42:19.238219 29932 leader_election.cc:288] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 712d32df021447a2832043d7665e0c59 (127.31.59.195:37201), b7841ae2f1704cb19f592e470376ddbe (127.31.59.193:41267)
I20260810 06:42:19.244326 29936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:19.238604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:37201 (local address 127.31.59.194:50607)
0810 06:42:19.239194 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:19.239210 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:19.239365 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:19.239704 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:19.239709 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:19.239737 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:19.240121 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:19.240132 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:19.241112 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:19.241118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:19.242558 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:19.242573 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:19.243786 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:19.243791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:19.243912 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:19.243935 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 06:42:19.244004 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 06:42:19.244104 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":306,"threads_started":1}
I20260810 06:42:19.244407 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:19.238720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:50607 (local address 127.31.59.195:37201)
0810 06:42:19.238937 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:19.238943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:19.239307 (+ 364us) server_negotiation.cc:407] Connection header received
0810 06:42:19.239522 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:19.239527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:19.239592 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:19.239667 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:19.240293 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:19.240987 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:19.242768 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:19.243598 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:19.243764 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:19.243831 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:19.244052 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 06:42:19.244173 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:42:19.245388 29937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:19.239343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:41267 (local address 127.31.59.194:36247)
0810 06:42:19.240216 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:19.240230 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:19.240348 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:19.240664 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:19.240669 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:19.240693 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:19.240990 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:19.240998 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:19.241931 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:19.241935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:19.243321 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:19.243333 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:19.244763 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:19.244771 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:19.244933 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:19.244963 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 06:42:19.245060 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 06:42:19.245170 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":523,"threads_started":1}
I20260810 06:42:19.245478 29919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:19.239648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:36247 (local address 127.31.59.193:41267)
0810 06:42:19.239816 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:19.239822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:19.240323 (+ 501us) server_negotiation.cc:407] Connection header received
0810 06:42:19.240476 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:19.240480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:19.240550 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:19.240638 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:19.241131 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:19.241798 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:19.243512 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:19.244590 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:19.244740 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:19.244790 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:19.245099 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 06:42:19.245238 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 06:42:19.245601 29812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4daaef484214afbb5bad2b61f1f4705" candidate_uuid: "129c72857944432987f5f248f7735a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "712d32df021447a2832043d7665e0c59" is_pre_election: true
I20260810 06:42:19.246140 29812 raft_consensus.cc:2454] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 129c72857944432987f5f248f7735a86 in term 0.
I20260810 06:42:19.246438 29377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4daaef484214afbb5bad2b61f1f4705" candidate_uuid: "129c72857944432987f5f248f7735a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7841ae2f1704cb19f592e470376ddbe" is_pre_election: true
I20260810 06:42:19.246793 29485 leader_election.cc:302] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 129c72857944432987f5f248f7735a86, 712d32df021447a2832043d7665e0c59; no voters:
I20260810 06:42:19.247013 29377 raft_consensus.cc:2454] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 129c72857944432987f5f248f7735a86 in term 0.
I20260810 06:42:19.247215 29932 raft_consensus.cc:2781] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:42:19.247349 29932 raft_consensus.cc:482] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:42:19.247450 29932 raft_consensus.cc:3037] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:19.249593 29932 raft_consensus.cc:504] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129c72857944432987f5f248f7735a86" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 41291 } } peers { permanent_uuid: "712d32df021447a2832043d7665e0c59" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37201 } } peers { permanent_uuid: "b7841ae2f1704cb19f592e470376ddbe" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 41267 } }
I20260810 06:42:19.250126 29932 leader_election.cc:288] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [CANDIDATE]: Term 1 election: Requested vote from peers 712d32df021447a2832043d7665e0c59 (127.31.59.195:37201), b7841ae2f1704cb19f592e470376ddbe (127.31.59.193:41267)
I20260810 06:42:19.250531 29812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4daaef484214afbb5bad2b61f1f4705" candidate_uuid: "129c72857944432987f5f248f7735a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "712d32df021447a2832043d7665e0c59"
I20260810 06:42:19.250592 29377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4daaef484214afbb5bad2b61f1f4705" candidate_uuid: "129c72857944432987f5f248f7735a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7841ae2f1704cb19f592e470376ddbe"
I20260810 06:42:19.250694 29812 raft_consensus.cc:3037] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:19.250777 29377 raft_consensus.cc:3037] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:19.252771 29812 raft_consensus.cc:2454] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 129c72857944432987f5f248f7735a86 in term 1.
I20260810 06:42:19.252923 29377 raft_consensus.cc:2454] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 129c72857944432987f5f248f7735a86 in term 1.
I20260810 06:42:19.253110 29485 leader_election.cc:302] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 129c72857944432987f5f248f7735a86, 712d32df021447a2832043d7665e0c59; no voters:
I20260810 06:42:19.253381 29932 raft_consensus.cc:2781] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:19.253798 29932 raft_consensus.cc:686] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [term 1 LEADER]: Becoming Leader. State: Replica: 129c72857944432987f5f248f7735a86, State: Running, Role: LEADER
I20260810 06:42:19.254233 29932 consensus_queue.cc:237] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129c72857944432987f5f248f7735a86" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 41291 } } peers { permanent_uuid: "712d32df021447a2832043d7665e0c59" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37201 } } peers { permanent_uuid: "b7841ae2f1704cb19f592e470376ddbe" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 41267 } }
I20260810 06:42:19.258677 29158 catalog_manager.cc:5495] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 129c72857944432987f5f248f7735a86 (127.31.59.194). New cstate: current_term: 1 leader_uuid: "129c72857944432987f5f248f7735a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129c72857944432987f5f248f7735a86" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 41291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "712d32df021447a2832043d7665e0c59" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7841ae2f1704cb19f592e470376ddbe" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 41267 } health_report { overall_health: UNKNOWN } } }
I20260810 06:42:19.336181 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:19.330276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60068 (local address 127.31.59.194:41291)
0810 06:42:19.330476 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:19.330482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:19.330544 (+ 62us) server_negotiation.cc:407] Connection header received
0810 06:42:19.330737 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:19.330742 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:19.330833 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:19.330959 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:42:19.331351 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:19.332028 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:19.334029 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:19.334529 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:19.334669 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:19.335152 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:42:19.335649 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:42:19.335768 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:19.335860 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 06:42:19.335930 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:42:19.344673 29812 raft_consensus.cc:1261] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59 [term 1 FOLLOWER]: Refusing update from remote peer 129c72857944432987f5f248f7735a86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:42:19.344710 29377 raft_consensus.cc:1261] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe [term 1 FOLLOWER]: Refusing update from remote peer 129c72857944432987f5f248f7735a86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:42:19.345608 29938 consensus_queue.cc:1035] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7841ae2f1704cb19f592e470376ddbe" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 41267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:42:19.345949 29932 consensus_queue.cc:1035] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "712d32df021447a2832043d7665e0c59" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:42:19.377782 29157 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:42:19.378077 29157 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:42:19.381788 29157 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f67b4d6e3ee147e690da6ed3cb7db7d0: Sending DeleteTablet for 3 replicas of tablet b4daaef484214afbb5bad2b61f1f4705
I20260810 06:42:19.382958 29345 tablet_service.cc:1514] Processing DeleteTablet for tablet b4daaef484214afbb5bad2b61f1f4705 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:42:19 UTC) from {username='slave'} at 127.0.0.1:55960
I20260810 06:42:19.382939 29779 tablet_service.cc:1514] Processing DeleteTablet for tablet b4daaef484214afbb5bad2b61f1f4705 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:42:19 UTC) from {username='slave'} at 127.0.0.1:60316
I20260810 06:42:19.385623 29559 tablet_service.cc:1514] Processing DeleteTablet for tablet b4daaef484214afbb5bad2b61f1f4705 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:42:19 UTC) from {username='slave'} at 127.0.0.1:60058
I20260810 06:42:19.387571 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 29250
I20260810 06:42:19.396445 29963 tablet_replica.cc:330] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86: stopping tablet replica
I20260810 06:42:19.397008 29963 raft_consensus.cc:2229] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:42:19.397862 29963 raft_consensus.cc:2258] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:42:19.398478 29962 tablet_replica.cc:330] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe: stopping tablet replica
I20260810 06:42:19.398896 29962 raft_consensus.cc:2229] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:42:19.399410 29962 raft_consensus.cc:2258] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:42:19.400385 29963 ts_tablet_manager.cc:1905] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:42:19.401612 29962 ts_tablet_manager.cc:1905] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:42:19.406116 29961 tablet_replica.cc:330] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59: stopping tablet replica
I20260810 06:42:19.408079 29963 ts_tablet_manager.cc:1918] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:42:19.408308 29963 log.cc:1196] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/b4daaef484214afbb5bad2b61f1f4705
I20260810 06:42:19.408423 29919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:19.388876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55966 (local address 127.31.59.193:41267)
0810 06:42:19.389073 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:19.389079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:19.394474 (+ 5395us) server_negotiation.cc:407] Connection header received
0810 06:42:19.394669 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:19.394675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:19.394830 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:19.394935 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:19.399748 (+ 4813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:19.400883 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:19.404500 (+ 3617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:19.405137 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:19.405253 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:19.406755 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:19.406765 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:19.406785 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:19.407547 (+ 762us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:19.407604 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:19.407629 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:19.407737 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:19.407953 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 06:42:19.408071 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:42:19.408767 29962 ts_tablet_manager.cc:1918] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:42:19.408926 29962 log.cc:1196] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/b4daaef484214afbb5bad2b61f1f4705
I20260810 06:42:19.409312 29963 ts_tablet_manager.cc:1939] T b4daaef484214afbb5bad2b61f1f4705 P 129c72857944432987f5f248f7735a86: Deleting consensus metadata
I20260810 06:42:19.409529 29442 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:19.410848 29961 raft_consensus.cc:2229] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:42:19.411491 29961 raft_consensus.cc:2258] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:42:19.411682 29135 catalog_manager.cc:4841] TS 129c72857944432987f5f248f7735a86 (127.31.59.194:41291): tablet b4daaef484214afbb5bad2b61f1f4705 (table test-table [id=48ec9bc7d28e43b8b86257dee1e2281c]) successfully deleted
I20260810 06:42:19.413803 29961 ts_tablet_manager.cc:1905] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:42:19.420940 29961 ts_tablet_manager.cc:1918] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:42:19.421119 29961 log.cc:1196] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/b4daaef484214afbb5bad2b61f1f4705
I20260810 06:42:19.421653 29961 ts_tablet_manager.cc:1939] T b4daaef484214afbb5bad2b61f1f4705 P 712d32df021447a2832043d7665e0c59: Deleting consensus metadata
I20260810 06:42:19.423534 29137 catalog_manager.cc:4841] TS 712d32df021447a2832043d7665e0c59 (127.31.59.195:37201): tablet b4daaef484214afbb5bad2b61f1f4705 (table test-table [id=48ec9bc7d28e43b8b86257dee1e2281c]) successfully deleted
I20260810 06:42:19.500845 29962 ts_tablet_manager.cc:1939] T b4daaef484214afbb5bad2b61f1f4705 P b7841ae2f1704cb19f592e470376ddbe: Deleting consensus metadata
I20260810 06:42:19.501439 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 29250
W20260810 06:42:19.528931 29137 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.59.193:41267 (error 108)
I20260810 06:42:19.529641 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 29457
I20260810 06:42:19.531462 29969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:19.530238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:41267 (local address 127.0.0.1:55972)
0810 06:42:19.531126 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:19.531265 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":762,"thread_start_us":665,"threads_started":1}
W20260810 06:42:19.531740 29137 catalog_manager.cc:4584] TS b7841ae2f1704cb19f592e470376ddbe (127.31.59.193:41267): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b4daaef484214afbb5bad2b61f1f4705: Network error: Client connection negotiation failed: client connection to 127.31.59.193:41267: connect: Connection refused (error 111)
I20260810 06:42:19.537410 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:19.530580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60078 (local address 127.31.59.194:41291)
0810 06:42:19.530765 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:19.530771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:19.530803 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:42:19.530870 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:19.530876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:19.530980 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:19.531109 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:19.531663 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:19.532755 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:19.534081 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:19.534793 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:19.534927 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:19.535845 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:19.535856 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:19.535875 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:19.536656 (+ 781us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:19.536720 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:19.536747 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:19.536863 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:19.536914 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 06:42:19.537032 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 06:42:19.538110 29660 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:19.585081 29969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:19.584639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:41267 (local address 127.0.0.1:55974)
0810 06:42:19.584825 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:19.584910 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 06:42:19.586083 29969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:19.585707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:41267 (local address 127.0.0.1:55976)
0810 06:42:19.585852 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:19.585944 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 06:42:19.647097 29969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:19.646624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:41267 (local address 127.0.0.1:55990)
0810 06:42:19.646813 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:19.646912 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 06:42:19.647111 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 29457
I20260810 06:42:19.655541 29969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:19.655109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:41267 (local address 127.0.0.1:56002)
0810 06:42:19.655300 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:19.655380 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 06:42:19.670106 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 29676
I20260810 06:42:19.677316 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:19.671042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60326 (local address 127.31.59.195:37201)
0810 06:42:19.671232 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:19.671238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:19.671260 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:42:19.671433 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:19.671439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:19.671545 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:19.671644 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:19.672085 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:19.672831 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:19.674192 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:19.674686 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:19.674815 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:19.675938 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:19.675948 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:19.675965 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:19.676544 (+ 579us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:19.676592 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:19.676613 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:19.676724 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:19.676861 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 06:42:19.676978 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:42:19.677892 29881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:19.735992 29969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:19.735498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:41267 (local address 127.0.0.1:56008)
0810 06:42:19.735762 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:19.735853 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260810 06:42:19.737123 29969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:19.736706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:41267 (local address 127.0.0.1:56020)
0810 06:42:19.736901 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:19.736973 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260810 06:42:19.763654 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 29676
I20260810 06:42:19.788164 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 29109
I20260810 06:42:19.789291 29206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:19.894707 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 29109
2026-08-10T06:42:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3084 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:42:20 dist-test-slave-4tdj krb5kdc[29979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:42:20 dist-test-slave-4tdj krb5kdc[29979](info): set up 2 sockets
Aug 10 06:42:20 dist-test-slave-4tdj krb5kdc[29979](info): commencing operation
krb5kdc: starting...
W20260810 06:42:21.140412 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.004s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:42:21 dist-test-slave-4tdj krb5kdc[29979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:42:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:42:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:42:21.397044 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40191
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:43719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:40191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:21.735888 30004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:21.736214 30004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:21.736368 30004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:21.743781 30004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:42:21.743903 30004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:21.743990 30004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:21.744081 30004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:42:21.744148 30004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:42:21.754073 30004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:43719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:40191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:40191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:21.755396 30004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:21.756732 30004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:21.768460 30012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:21.768473 30014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:21.768556 30010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:21.769361 30004 server_base.cc:1034] running on GCE node
I20260810 06:42:21.770133 30004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:21.771539 30004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:21.772758 30004 hybrid_clock.cc:648] HybridClock initialized: now 1786344141772696 us; error 44 us; skew 500 ppm
I20260810 06:42:21.773082 30004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:21 dist-test-slave-4tdj krb5kdc[29979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344141, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:21.776085 30004 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:42:21.777595 30004 webserver.cc:460] Webserver started at http://127.31.59.254:45077/ using document root <none> and password file <none>
I20260810 06:42:21.778247 30004 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:21.778410 30004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:21.778730 30004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:21.781735 30004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "edb8f7c2118c4508b5fca88e363dfd81"
format_stamp: "Formatted at 2026-08-10 06:42:21 on dist-test-slave-4tdj"
server_key: "b5a0ac5db750e50ac3abdca5ba8bcdc9"
server_key_iv: "b008b88370ad4ce95c47da5a26c09f8f"
server_key_version: "encryptionkey@0"
I20260810 06:42:21.782564 30004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "edb8f7c2118c4508b5fca88e363dfd81"
format_stamp: "Formatted at 2026-08-10 06:42:21 on dist-test-slave-4tdj"
server_key: "b5a0ac5db750e50ac3abdca5ba8bcdc9"
server_key_iv: "b008b88370ad4ce95c47da5a26c09f8f"
server_key_version: "encryptionkey@0"
I20260810 06:42:21.787717 30004 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:42:21.791159 30034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:21.792454 30004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:42:21.792646 30004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "edb8f7c2118c4508b5fca88e363dfd81"
format_stamp: "Formatted at 2026-08-10 06:42:21 on dist-test-slave-4tdj"
server_key: "b5a0ac5db750e50ac3abdca5ba8bcdc9"
server_key_iv: "b008b88370ad4ce95c47da5a26c09f8f"
server_key_version: "encryptionkey@0"
I20260810 06:42:21.792816 30004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:21.810101 30004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:21.813824 30004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:21.814074 30004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:21.831689 30004 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:40191
I20260810 06:42:21.831708 30121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:40191 every 8 connection(s)
I20260810 06:42:21.833031 30004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:42:21.836632 30122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:21.842681 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 30004
I20260810 06:42:21.842859 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:42:21.843309 31983 external_mini_cluster.cc:1428] Setting key 9f8a86779d7acf20e981f68f90a1e7e3
I20260810 06:42:21.847012 30122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81: Bootstrap starting.
I20260810 06:42:21.850498 30122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:21.851516 30122 log.cc:824] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:21.854007 30122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81: No bootstrap required, opened a new log
Aug 10 06:42:21 dist-test-slave-4tdj krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:42:21.862234 30122 raft_consensus.cc:348] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edb8f7c2118c4508b5fca88e363dfd81" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40191 } }
I20260810 06:42:21.862711 30122 raft_consensus.cc:374] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:21.862813 30122 raft_consensus.cc:729] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edb8f7c2118c4508b5fca88e363dfd81, State: Initialized, Role: FOLLOWER
I20260810 06:42:21.863498 30122 consensus_queue.cc:260] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edb8f7c2118c4508b5fca88e363dfd81" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40191 } }
I20260810 06:42:21.863744 30122 raft_consensus.cc:388] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:42:21.863883 30122 raft_consensus.cc:482] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:42:21.864037 30122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:21.866358 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:21.845225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47588 (local address 127.31.59.254:40191)
0810 06:42:21.846026 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:21.846038 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:21.846081 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:42:21.847933 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:21.847956 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:21.848332 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:21.848670 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:21.850449 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:21.851852 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:21.853242 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:21.853919 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:21.854124 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:21.858713 (+ 4589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:21.858721 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:21.858734 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:21.858780 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:21.861846 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:21.862734 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:21.862741 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:21.862748 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:21.862997 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:21.863431 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:21.863435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:21.863437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:21.864123 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:21.864383 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:21.864825 (+ 442us) server_negotiation.cc:299] Negotiation successful
0810 06:42:21.865253 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":354,"threads_started":1}
I20260810 06:42:21.866683 30122 raft_consensus.cc:504] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edb8f7c2118c4508b5fca88e363dfd81" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40191 } }
I20260810 06:42:21.867158 30122 leader_election.cc:302] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edb8f7c2118c4508b5fca88e363dfd81; no voters:
I20260810 06:42:21.867730 30122 leader_election.cc:288] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:42:21.867883 30130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:21.869050 30130 raft_consensus.cc:686] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81 [term 1 LEADER]: Becoming Leader. State: Replica: edb8f7c2118c4508b5fca88e363dfd81, State: Running, Role: LEADER
I20260810 06:42:21.869711 30130 consensus_queue.cc:237] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edb8f7c2118c4508b5fca88e363dfd81" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40191 } }
I20260810 06:42:21.869850 30122 sys_catalog.cc:564] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:42:21.872547 30135 sys_catalog.cc:455] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edb8f7c2118c4508b5fca88e363dfd81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edb8f7c2118c4508b5fca88e363dfd81" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40191 } } }
I20260810 06:42:21.872987 30135 sys_catalog.cc:458] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:21.872866 30138 sys_catalog.cc:455] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edb8f7c2118c4508b5fca88e363dfd81. Latest consensus state: current_term: 1 leader_uuid: "edb8f7c2118c4508b5fca88e363dfd81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edb8f7c2118c4508b5fca88e363dfd81" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 40191 } } }
I20260810 06:42:21.873172 30138 sys_catalog.cc:458] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:21.875416 30145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:42:21.879721 30145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:42:21.888531 30145 catalog_manager.cc:1347] Generated new cluster ID: 997388ac7db6477da6ca718706b4cf15
I20260810 06:42:21.888648 30145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:42:21.896104 30145 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:42:21.897552 30145 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:42:21.908766 30145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81: Generated new TSK 0
I20260810 06:42:21.909507 30145 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
I20260810 06:42:21.973028 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40191
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:43719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:22.303646 30170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:22.303916 30170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:22.304008 30170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:22.311069 30170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:22.311172 30170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:22.311287 30170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:42:22.320763 30170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:43719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:22.321944 30170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:22.323230 30170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:22.335937 30177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:22.335896 30181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:22.335863 30178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:22.336887 30170 server_base.cc:1034] running on GCE node
I20260810 06:42:22.337452 30170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:22.338370 30170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:22.339578 30170 hybrid_clock.cc:648] HybridClock initialized: now 1786344142339502 us; error 66 us; skew 500 ppm
I20260810 06:42:22.339874 30170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:22 dist-test-slave-4tdj krb5kdc[29979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344142, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:22.346807 30170 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:42:22.348188 30170 webserver.cc:460] Webserver started at http://127.31.59.193:46403/ using document root <none> and password file <none>
I20260810 06:42:22.348882 30170 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:22.349000 30170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:22.349318 30170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:22.352178 30170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "6a6df51292f04b02a5b791b349ce0a2c"
format_stamp: "Formatted at 2026-08-10 06:42:22 on dist-test-slave-4tdj"
server_key: "b38fd8d4fed969279deaba627a9037c0"
server_key_iv: "05ac899ac9ea33a2ef23c8d68dc4eedd"
server_key_version: "encryptionkey@0"
I20260810 06:42:22.352909 30170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "6a6df51292f04b02a5b791b349ce0a2c"
format_stamp: "Formatted at 2026-08-10 06:42:22 on dist-test-slave-4tdj"
server_key: "b38fd8d4fed969279deaba627a9037c0"
server_key_iv: "05ac899ac9ea33a2ef23c8d68dc4eedd"
server_key_version: "encryptionkey@0"
I20260810 06:42:22.357889 30170 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 06:42:22.361297 30192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:22.362859 30170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:42:22.363029 30170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "6a6df51292f04b02a5b791b349ce0a2c"
format_stamp: "Formatted at 2026-08-10 06:42:22 on dist-test-slave-4tdj"
server_key: "b38fd8d4fed969279deaba627a9037c0"
server_key_iv: "05ac899ac9ea33a2ef23c8d68dc4eedd"
server_key_version: "encryptionkey@0"
I20260810 06:42:22.363237 30170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:22.396504 30170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:22.400583 30170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:22.400879 30170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:22.401878 30170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:42:22.403285 30170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:42:22.403378 30170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:22.403529 30170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:42:22.403605 30170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:22.435459 30170 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:45763
I20260810 06:42:22.435491 30387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:45763 every 8 connection(s)
I20260810 06:42:22.436842 30170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:42:22.440407 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 30170
I20260810 06:42:22.440610 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:42:22.441217 31983 external_mini_cluster.cc:1428] Setting key 99a5f2fed4f3430db7c0904850ba1dea
Aug 10 06:42:22 dist-test-slave-4tdj krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344142, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:42:22.457216 30394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:22.439365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:45527 (local address 127.31.59.254:40191)
0810 06:42:22.439812 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:22.439816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:22.440925 (+ 1109us) server_negotiation.cc:407] Connection header received
0810 06:42:22.443082 (+ 2157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:22.443087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:22.443166 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:22.443251 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:22.444736 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:22.445755 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:22.447423 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:22.447968 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:22.448093 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:22.451398 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:22.451404 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:22.451407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:22.451444 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:22.454375 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:22.455074 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:22.455079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:22.455083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:22.455204 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:22.455656 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:22.455659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:22.455661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:22.455897 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:22.456002 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:22.456803 (+ 801us) server_negotiation.cc:299] Negotiation successful
0810 06:42:22.456968 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":231,"threads_started":1}
I20260810 06:42:22.458527 30395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:22.439962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40191 (local address 127.31.59.193:45527)
0810 06:42:22.440771 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:22.440817 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:22.442785 (+ 1968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:22.443483 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:22.443492 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:22.443978 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:42:22.444560 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:22.444579 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:22.445943 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:22.445948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:22.447278 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:22.447292 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:22.448156 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:22.448161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:22.448352 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:22.448968 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:42:22.448998 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:42:22.451107 (+ 2109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:42:22.454515 (+ 3408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:42:22.454522 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:42:22.454534 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:42:22.454942 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:42:22.455300 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:42:22.455305 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:42:22.455308 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:42:22.455560 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:42:22.456009 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:42:22.456016 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:42:22.456439 (+ 423us) client_negotiation.cc:769] Sending connection context
0810 06:42:22.456726 (+ 287us) client_negotiation.cc:240] Negotiation successful
0810 06:42:22.457206 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":351,"threads_started":1}
I20260810 06:42:22.460079 30392 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40191
I20260810 06:42:22.460393 30392 heartbeater.cc:464] Registering TS with master...
I20260810 06:42:22.461215 30392 heartbeater.cc:511] Master 127.31.59.254:40191 requested a full tablet report, sending...
I20260810 06:42:22.463665 30066 ts_manager.cc:194] Registered new tserver with Master: 6a6df51292f04b02a5b791b349ce0a2c (127.31.59.193:45763)
I20260810 06:42:22.465570 30066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:45527
WARNING: no policy specified for kudu/127.31.59.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
I20260810 06:42:22.505954 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40191
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:43719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:22.837913 30404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:22.838234 30404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:22.838406 30404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:22.849514 30404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:22.849653 30404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:22.849831 30404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:42:22.860814 30404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:43719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:22.861922 30404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:22.863175 30404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:22.875178 30418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:22.875231 30417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:22.875257 30422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:22.876230 30404 server_base.cc:1034] running on GCE node
I20260810 06:42:22.876776 30404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:22.877635 30404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:22.878823 30404 hybrid_clock.cc:648] HybridClock initialized: now 1786344142878775 us; error 37 us; skew 500 ppm
I20260810 06:42:22.879112 30404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:22 dist-test-slave-4tdj krb5kdc[29979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344142, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:22.881966 30404 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:42:22.883286 30404 webserver.cc:460] Webserver started at http://127.31.59.194:43747/ using document root <none> and password file <none>
I20260810 06:42:22.883903 30404 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:22.884011 30404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:22.884305 30404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:22.887072 30404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "975603f9b663482cba3f87b6745f189d"
format_stamp: "Formatted at 2026-08-10 06:42:22 on dist-test-slave-4tdj"
server_key: "991cda749462e4c6be53420a36738348"
server_key_iv: "d3722027a72dd18c01afd0e28b282bcb"
server_key_version: "encryptionkey@0"
I20260810 06:42:22.887770 30404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "975603f9b663482cba3f87b6745f189d"
format_stamp: "Formatted at 2026-08-10 06:42:22 on dist-test-slave-4tdj"
server_key: "991cda749462e4c6be53420a36738348"
server_key_iv: "d3722027a72dd18c01afd0e28b282bcb"
server_key_version: "encryptionkey@0"
I20260810 06:42:22.892596 30404 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 06:42:22.895900 30430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:22.897240 30404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:42:22.897400 30404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "975603f9b663482cba3f87b6745f189d"
format_stamp: "Formatted at 2026-08-10 06:42:22 on dist-test-slave-4tdj"
server_key: "991cda749462e4c6be53420a36738348"
server_key_iv: "d3722027a72dd18c01afd0e28b282bcb"
server_key_version: "encryptionkey@0"
I20260810 06:42:22.897565 30404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:22.916433 30404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:22.920185 30404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:22.920426 30404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:22.921322 30404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:42:22.922686 30404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:42:22.922758 30404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:22.922864 30404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:42:22.922911 30404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:22.952455 30404 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:38283
I20260810 06:42:22.952500 30640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:38283 every 8 connection(s)
I20260810 06:42:22.953696 30404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:42:22.963019 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 30404
I20260810 06:42:22.963202 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:42:22.963781 31983 external_mini_cluster.cc:1428] Setting key b336f05ebe48ceec947968201c59a962
Aug 10 06:42:22 dist-test-slave-4tdj krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344142, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:42:22.974069 30394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:22.956150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:60507 (local address 127.31.59.254:40191)
0810 06:42:22.956324 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:22.956328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:22.957455 (+ 1127us) server_negotiation.cc:407] Connection header received
0810 06:42:22.959258 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:22.959264 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:22.959345 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:22.959431 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:22.960930 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:22.961640 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:22.963233 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:22.963882 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:22.964028 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:22.968118 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:22.968125 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:22.968130 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:22.968171 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:22.971138 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:22.971854 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:22.971859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:22.971862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:22.972017 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:22.972400 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:22.972404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:22.972407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:22.972721 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:22.972853 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:22.973521 (+ 668us) server_negotiation.cc:299] Negotiation successful
0810 06:42:22.973736 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:42:22.974939 30646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:22.956595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40191 (local address 127.31.59.194:60507)
0810 06:42:22.957288 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:22.957332 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:22.958969 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:22.959669 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:22.959680 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:22.960197 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:42:22.960754 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:22.960771 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:22.961788 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:22.961792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:22.963064 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:22.963076 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:22.964095 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:22.964100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:22.964322 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:22.965068 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:42:22.965104 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:42:22.967854 (+ 2750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:42:22.971273 (+ 3419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:42:22.971279 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:42:22.971294 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:42:22.971731 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:42:22.972118 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:42:22.972121 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:42:22.972123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:42:22.972299 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:42:22.972829 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:42:22.972834 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:42:22.973155 (+ 321us) client_negotiation.cc:769] Sending connection context
0810 06:42:22.973383 (+ 228us) client_negotiation.cc:240] Negotiation successful
0810 06:42:22.973781 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":256,"threads_started":1}
I20260810 06:42:22.976257 30642 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40191
I20260810 06:42:22.976533 30642 heartbeater.cc:464] Registering TS with master...
I20260810 06:42:22.977248 30642 heartbeater.cc:511] Master 127.31.59.254:40191 requested a full tablet report, sending...
I20260810 06:42:22.978749 30066 ts_manager.cc:194] Registered new tserver with Master: 975603f9b663482cba3f87b6745f189d (127.31.59.194:38283)
I20260810 06:42:22.979786 30066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:60507
WARNING: no policy specified for kudu/127.31.59.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
I20260810 06:42:23.027123 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40191
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:43719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:23.356122 30649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:23.356405 30649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:23.356550 30649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:23.363492 30649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:23.363601 30649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:23.363760 30649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:42:23.374738 30649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:43719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:40191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:23.375952 30649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:23.377224 30649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:23.389684 30663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:23.389724 30658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:23.389650 30659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:23.390486 30649 server_base.cc:1034] running on GCE node
I20260810 06:42:23.391098 30649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:23.392683 30649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:23.393874 30649 hybrid_clock.cc:648] HybridClock initialized: now 1786344143393814 us; error 51 us; skew 500 ppm
I20260810 06:42:23.394145 30649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:23 dist-test-slave-4tdj krb5kdc[29979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344143, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:23.397294 30649 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:42:23.398718 30649 webserver.cc:460] Webserver started at http://127.31.59.195:35237/ using document root <none> and password file <none>
I20260810 06:42:23.399350 30649 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:23.399463 30649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:23.399746 30649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:23.402650 30649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "8b9214ae009a4eb9ae7d135f4910154a"
format_stamp: "Formatted at 2026-08-10 06:42:23 on dist-test-slave-4tdj"
server_key: "05df47bf4b035522fa3a5977dcacc1ef"
server_key_iv: "f29975f457932c27284fd21221f59c4f"
server_key_version: "encryptionkey@0"
I20260810 06:42:23.403378 30649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "8b9214ae009a4eb9ae7d135f4910154a"
format_stamp: "Formatted at 2026-08-10 06:42:23 on dist-test-slave-4tdj"
server_key: "05df47bf4b035522fa3a5977dcacc1ef"
server_key_iv: "f29975f457932c27284fd21221f59c4f"
server_key_version: "encryptionkey@0"
I20260810 06:42:23.408457 30649 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:42:23.413056 30671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:23.414848 30649 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 06:42:23.415081 30649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "8b9214ae009a4eb9ae7d135f4910154a"
format_stamp: "Formatted at 2026-08-10 06:42:23 on dist-test-slave-4tdj"
server_key: "05df47bf4b035522fa3a5977dcacc1ef"
server_key_iv: "f29975f457932c27284fd21221f59c4f"
server_key_version: "encryptionkey@0"
I20260810 06:42:23.415380 30649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:23.432344 30649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:23.436349 30649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:23.436614 30649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:23.437541 30649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:42:23.438900 30649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:42:23.438977 30649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:23.439098 30649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:42:23.439149 30649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:23.468403 30649 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:42573
I20260810 06:42:23.468468 30888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:42573 every 8 connection(s)
I20260810 06:42:23.469137 30392 heartbeater.cc:503] Master 127.31.59.254:40191 was elected leader, sending a full tablet report...
I20260810 06:42:23.469874 30649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:42:23.478741 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 30649
I20260810 06:42:23.478932 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:42:23.479493 31983 external_mini_cluster.cc:1428] Setting key 2ff56d9561297f08d010735df686ebc5
Aug 10 06:42:23 dist-test-slave-4tdj krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344143, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:42:23.489611 30394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:23.472454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:41873 (local address 127.31.59.254:40191)
0810 06:42:23.472632 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:23.472636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:23.473802 (+ 1166us) server_negotiation.cc:407] Connection header received
0810 06:42:23.475574 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:23.475579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:23.475643 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:23.475727 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:23.477363 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.478372 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.480414 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.480875 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.480987 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:23.484432 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:23.484437 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:23.484440 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:23.484471 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:23.486876 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:23.487520 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:23.487524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:23.487526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:23.487655 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:23.488041 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:23.488046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:23.488049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:23.488305 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:23.488443 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:23.489068 (+ 625us) server_negotiation.cc:299] Negotiation successful
0810 06:42:23.489274 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:42:23.490517 30892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:23.472946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:40191 (local address 127.31.59.195:41873)
0810 06:42:23.473645 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:23.473685 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:23.475311 (+ 1626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:23.475994 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:23.476002 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:23.476497 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:42:23.477154 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:23.477182 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.478578 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.478583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:23.480276 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:23.480291 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.481111 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.481119 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:23.481340 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:23.482027 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:42:23.482055 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:42:23.484245 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:42:23.486998 (+ 2753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:42:23.487007 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:42:23.487022 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:42:23.487411 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:42:23.487758 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:42:23.487763 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:42:23.487766 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:42:23.487938 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:42:23.488426 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:42:23.488434 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:42:23.488728 (+ 294us) client_negotiation.cc:769] Sending connection context
0810 06:42:23.488955 (+ 227us) client_negotiation.cc:240] Negotiation successful
0810 06:42:23.489427 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":258,"threads_started":1}
I20260810 06:42:23.491721 30889 heartbeater.cc:347] Connected to a master server at 127.31.59.254:40191
I20260810 06:42:23.491981 30889 heartbeater.cc:464] Registering TS with master...
I20260810 06:42:23.492765 30889 heartbeater.cc:511] Master 127.31.59.254:40191 requested a full tablet report, sending...
I20260810 06:42:23.494206 30065 ts_manager.cc:194] Registered new tserver with Master: 8b9214ae009a4eb9ae7d135f4910154a (127.31.59.195:42573)
I20260810 06:42:23.495081 30065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:41873
I20260810 06:42:23.506326 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:42:23.522836 30394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:23.510341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47604 (local address 127.31.59.254:40191)
0810 06:42:23.510498 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:23.510503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:23.510960 (+ 457us) server_negotiation.cc:407] Connection header received
0810 06:42:23.511106 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:23.511110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:23.511174 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:23.511248 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:23.512508 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.513215 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.514635 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.515062 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.515155 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:23.517745 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:23.517750 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:23.517753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:23.517783 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:23.520326 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:23.521145 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:23.521149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:23.521152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:23.521282 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:23.521649 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:23.521653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:23.521655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:23.521954 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:23.522167 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:23.522480 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 06:42:23.522616 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:42:23.526805 30065 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:42:23.533597 30065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:42:23.563418 30915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:23.545301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:42573 (local address 127.0.0.1:33054)
0810 06:42:23.555265 (+ 9964us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:23.555284 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:23.555407 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:23.556019 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:23.556026 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:23.556066 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:23.556483 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:23.556499 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.558064 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.558069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:23.560487 (+ 2418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:23.560502 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.562435 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.562441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:23.562663 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:23.562720 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 06:42:23.562853 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 06:42:23.562979 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9814,"mutex_wait_us":3,"thread_start_us":8946,"threads_started":1}
I20260810 06:42:23.563944 30912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:23.545545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:38283 (local address 127.0.0.1:50034)
0810 06:42:23.556147 (+ 10602us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:23.556166 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:23.556287 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:23.556912 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:23.556917 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:23.556941 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:23.557319 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:23.557330 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.559545 (+ 2215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.559550 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:23.561382 (+ 1832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:23.561397 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.563197 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.563203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:23.563388 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:23.563425 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 06:42:23.563511 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 06:42:23.563627 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10215,"thread_start_us":9423,"threads_started":1}
I20260810 06:42:23.564016 30914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:23.544876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45763 (local address 127.0.0.1:49700)
0810 06:42:23.554594 (+ 9718us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:23.554631 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:23.554786 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:23.556785 (+ 1999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:23.556790 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:23.556815 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:23.557277 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:23.557295 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.559262 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.559267 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:23.561110 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:23.561122 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.562844 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.562851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:23.563005 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:23.563035 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 06:42:23.563113 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 06:42:23.563245 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9481,"thread_start_us":10753,"threads_started":1}
I20260810 06:42:23.564261 30913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:23.545247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49700 (local address 127.31.59.193:45763)
0810 06:42:23.556230 (+ 10983us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:23.556237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:23.556262 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:42:23.556355 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:23.556363 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:23.556602 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:23.556790 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:23.557473 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.559076 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.561250 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.562602 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.563501 (+ 899us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:23.563715 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:23.563778 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 06:42:23.563932 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10826,"thread_start_us":10984,"threads_started":1}
I20260810 06:42:23.564802 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:23.546221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33054 (local address 127.31.59.195:42573)
0810 06:42:23.554443 (+ 8222us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:23.554453 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:23.555432 (+ 979us) server_negotiation.cc:407] Connection header received
0810 06:42:23.555542 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:23.555560 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:23.555816 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:23.556001 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:23.556647 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.557939 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.560653 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.562012 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.562130 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:23.564353 (+ 2223us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:23.564425 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 06:42:23.564573 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8030,"thread_start_us":7877,"threads_started":1}
I20260810 06:42:23.565202 30917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:23.546726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50034 (local address 127.31.59.194:38283)
0810 06:42:23.554850 (+ 8124us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:23.554860 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:23.556410 (+ 1550us) server_negotiation.cc:407] Connection header received
0810 06:42:23.556508 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:23.556514 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:23.556709 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:23.557497 (+ 788us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:23.557722 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.559417 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.561563 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.562988 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.563124 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:23.563304 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:23.563911 (+ 607us) server_negotiation.cc:299] Negotiation successful
0810 06:42:23.564326 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7994,"thread_start_us":7863,"threads_started":1}
I20260810 06:42:23.568146 30759 tablet_service.cc:1467] Processing CreateTablet for tablet 9aab848994cd4997a9dd4cd8afffbae0 (DEFAULT_TABLE table=test-table [id=5fcf88fd4c214f73b15a3931b4167fdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:42:23.568243 30276 tablet_service.cc:1467] Processing CreateTablet for tablet 9aab848994cd4997a9dd4cd8afffbae0 (DEFAULT_TABLE table=test-table [id=5fcf88fd4c214f73b15a3931b4167fdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:42:23.568523 30526 tablet_service.cc:1467] Processing CreateTablet for tablet 9aab848994cd4997a9dd4cd8afffbae0 (DEFAULT_TABLE table=test-table [id=5fcf88fd4c214f73b15a3931b4167fdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:42:23.569566 30759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aab848994cd4997a9dd4cd8afffbae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:23.569671 30276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aab848994cd4997a9dd4cd8afffbae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:23.569697 30526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aab848994cd4997a9dd4cd8afffbae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:23.580852 30920 tablet_bootstrap.cc:492] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a: Bootstrap starting.
I20260810 06:42:23.583770 30924 tablet_bootstrap.cc:492] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c: Bootstrap starting.
I20260810 06:42:23.585522 30920 tablet_bootstrap.cc:654] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:23.586953 30921 tablet_bootstrap.cc:492] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d: Bootstrap starting.
I20260810 06:42:23.586952 30920 log.cc:824] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:23.587193 30924 tablet_bootstrap.cc:654] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:23.588271 30924 log.cc:824] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:23.590138 30920 tablet_bootstrap.cc:492] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a: No bootstrap required, opened a new log
I20260810 06:42:23.590162 30921 tablet_bootstrap.cc:654] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:23.590440 30920 ts_tablet_manager.cc:1397] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260810 06:42:23.591109 30921 log.cc:824] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:23.591179 30924 tablet_bootstrap.cc:492] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c: No bootstrap required, opened a new log
I20260810 06:42:23.591413 30924 ts_tablet_manager.cc:1397] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260810 06:42:23.597762 30921 tablet_bootstrap.cc:492] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d: No bootstrap required, opened a new log
I20260810 06:42:23.597982 30921 ts_tablet_manager.cc:1397] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.009s
I20260810 06:42:23.600595 30920 raft_consensus.cc:348] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6df51292f04b02a5b791b349ce0a2c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45763 } } peers { permanent_uuid: "8b9214ae009a4eb9ae7d135f4910154a" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42573 } } peers { permanent_uuid: "975603f9b663482cba3f87b6745f189d" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38283 } }
I20260810 06:42:23.601013 30920 raft_consensus.cc:374] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:23.601136 30920 raft_consensus.cc:729] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b9214ae009a4eb9ae7d135f4910154a, State: Initialized, Role: FOLLOWER
I20260810 06:42:23.601437 30924 raft_consensus.cc:348] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6df51292f04b02a5b791b349ce0a2c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45763 } } peers { permanent_uuid: "8b9214ae009a4eb9ae7d135f4910154a" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42573 } } peers { permanent_uuid: "975603f9b663482cba3f87b6745f189d" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38283 } }
I20260810 06:42:23.601905 30924 raft_consensus.cc:374] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:23.601749 30920 consensus_queue.cc:260] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6df51292f04b02a5b791b349ce0a2c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45763 } } peers { permanent_uuid: "8b9214ae009a4eb9ae7d135f4910154a" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42573 } } peers { permanent_uuid: "975603f9b663482cba3f87b6745f189d" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38283 } }
I20260810 06:42:23.602002 30924 raft_consensus.cc:729] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6df51292f04b02a5b791b349ce0a2c, State: Initialized, Role: FOLLOWER
I20260810 06:42:23.602897 30889 heartbeater.cc:503] Master 127.31.59.254:40191 was elected leader, sending a full tablet report...
I20260810 06:42:23.602752 30924 consensus_queue.cc:260] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6df51292f04b02a5b791b349ce0a2c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45763 } } peers { permanent_uuid: "8b9214ae009a4eb9ae7d135f4910154a" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42573 } } peers { permanent_uuid: "975603f9b663482cba3f87b6745f189d" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38283 } }
I20260810 06:42:23.603629 30920 ts_tablet_manager.cc:1428] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a: Time spent starting tablet: real 0.013s user 0.005s sys 0.007s
I20260810 06:42:23.605083 30924 ts_tablet_manager.cc:1428] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c: Time spent starting tablet: real 0.014s user 0.000s sys 0.014s
I20260810 06:42:23.606369 30921 raft_consensus.cc:348] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6df51292f04b02a5b791b349ce0a2c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45763 } } peers { permanent_uuid: "8b9214ae009a4eb9ae7d135f4910154a" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42573 } } peers { permanent_uuid: "975603f9b663482cba3f87b6745f189d" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38283 } }
I20260810 06:42:23.606735 30921 raft_consensus.cc:374] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:23.606820 30921 raft_consensus.cc:729] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 975603f9b663482cba3f87b6745f189d, State: Initialized, Role: FOLLOWER
I20260810 06:42:23.607362 30921 consensus_queue.cc:260] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6df51292f04b02a5b791b349ce0a2c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45763 } } peers { permanent_uuid: "8b9214ae009a4eb9ae7d135f4910154a" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42573 } } peers { permanent_uuid: "975603f9b663482cba3f87b6745f189d" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38283 } }
I20260810 06:42:23.608410 30642 heartbeater.cc:503] Master 127.31.59.254:40191 was elected leader, sending a full tablet report...
I20260810 06:42:23.609117 30921 ts_tablet_manager.cc:1428] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260810 06:42:23.660691 30934 raft_consensus.cc:482] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:42:23.661002 30934 raft_consensus.cc:504] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6df51292f04b02a5b791b349ce0a2c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45763 } } peers { permanent_uuid: "8b9214ae009a4eb9ae7d135f4910154a" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42573 } } peers { permanent_uuid: "975603f9b663482cba3f87b6745f189d" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38283 } }
I20260810 06:42:23.662775 30934 leader_election.cc:288] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a6df51292f04b02a5b791b349ce0a2c (127.31.59.193:45763), 975603f9b663482cba3f87b6745f189d (127.31.59.194:38283)
I20260810 06:42:23.668941 30913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:23.663302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:34007 (local address 127.31.59.193:45763)
0810 06:42:23.663482 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:23.663487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:23.663508 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:42:23.663679 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:23.663685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:23.663765 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:23.663867 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:23.664265 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.665207 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.667013 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.668079 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.668192 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:23.668244 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:23.668664 (+ 420us) server_negotiation.cc:299] Negotiation successful
0810 06:42:23.668757 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:42:23.669183 30892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:23.663065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45763 (local address 127.31.59.195:34007)
0810 06:42:23.663366 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:23.663388 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:23.663520 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:23.663893 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:23.663897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:23.663922 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:23.664118 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:23.664130 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.665352 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.665357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:23.666863 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:23.666876 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.668217 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.668223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:23.668406 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:23.668443 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 06:42:23.668580 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 06:42:23.668711 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160}
I20260810 06:42:23.669808 30939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:23.663169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:38283 (local address 127.31.59.195:55309)
0810 06:42:23.663944 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:23.663968 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:23.664091 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:23.664391 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:23.664395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:23.664420 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:23.664764 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:23.664775 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.665992 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.665996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:23.667679 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:23.667693 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.669158 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.669163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:23.669304 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:23.669337 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:42:23.669463 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 06:42:23.669557 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":408,"threads_started":1}
I20260810 06:42:23.669888 30917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:23.663211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:55309 (local address 127.31.59.194:38283)
0810 06:42:23.663409 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:23.663415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:23.664070 (+ 655us) server_negotiation.cc:407] Connection header received
0810 06:42:23.664216 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:23.664221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:23.664292 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:23.664358 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:23.664916 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.665837 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.667823 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.668842 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.668993 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:23.669062 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:23.669525 (+ 463us) server_negotiation.cc:299] Negotiation successful
0810 06:42:23.669650 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:42:23.670228 30311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aab848994cd4997a9dd4cd8afffbae0" candidate_uuid: "8b9214ae009a4eb9ae7d135f4910154a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6df51292f04b02a5b791b349ce0a2c" is_pre_election: true
I20260810 06:42:23.670783 30311 raft_consensus.cc:2454] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b9214ae009a4eb9ae7d135f4910154a in term 0.
I20260810 06:42:23.670799 30558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aab848994cd4997a9dd4cd8afffbae0" candidate_uuid: "8b9214ae009a4eb9ae7d135f4910154a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "975603f9b663482cba3f87b6745f189d" is_pre_election: true
I20260810 06:42:23.671355 30558 raft_consensus.cc:2454] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b9214ae009a4eb9ae7d135f4910154a in term 0.
I20260810 06:42:23.671479 30675 leader_election.cc:302] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a6df51292f04b02a5b791b349ce0a2c, 8b9214ae009a4eb9ae7d135f4910154a; no voters:
I20260810 06:42:23.671862 30934 raft_consensus.cc:2781] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:42:23.672019 30934 raft_consensus.cc:482] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:42:23.672134 30934 raft_consensus.cc:3037] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:23.674237 30934 raft_consensus.cc:504] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6df51292f04b02a5b791b349ce0a2c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45763 } } peers { permanent_uuid: "8b9214ae009a4eb9ae7d135f4910154a" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42573 } } peers { permanent_uuid: "975603f9b663482cba3f87b6745f189d" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38283 } }
I20260810 06:42:23.674787 30934 leader_election.cc:288] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [CANDIDATE]: Term 1 election: Requested vote from peers 6a6df51292f04b02a5b791b349ce0a2c (127.31.59.193:45763), 975603f9b663482cba3f87b6745f189d (127.31.59.194:38283)
I20260810 06:42:23.675202 30311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aab848994cd4997a9dd4cd8afffbae0" candidate_uuid: "8b9214ae009a4eb9ae7d135f4910154a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6df51292f04b02a5b791b349ce0a2c"
I20260810 06:42:23.675232 30558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aab848994cd4997a9dd4cd8afffbae0" candidate_uuid: "8b9214ae009a4eb9ae7d135f4910154a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "975603f9b663482cba3f87b6745f189d"
I20260810 06:42:23.675406 30558 raft_consensus.cc:3037] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:23.675406 30311 raft_consensus.cc:3037] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:23.678511 30311 raft_consensus.cc:2454] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b9214ae009a4eb9ae7d135f4910154a in term 1.
I20260810 06:42:23.678511 30558 raft_consensus.cc:2454] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b9214ae009a4eb9ae7d135f4910154a in term 1.
I20260810 06:42:23.678895 30673 leader_election.cc:302] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b9214ae009a4eb9ae7d135f4910154a, 975603f9b663482cba3f87b6745f189d; no voters:
I20260810 06:42:23.679219 30934 raft_consensus.cc:2781] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:23.679626 30934 raft_consensus.cc:686] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [term 1 LEADER]: Becoming Leader. State: Replica: 8b9214ae009a4eb9ae7d135f4910154a, State: Running, Role: LEADER
I20260810 06:42:23.680064 30934 consensus_queue.cc:237] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6df51292f04b02a5b791b349ce0a2c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45763 } } peers { permanent_uuid: "8b9214ae009a4eb9ae7d135f4910154a" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42573 } } peers { permanent_uuid: "975603f9b663482cba3f87b6745f189d" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38283 } }
I20260810 06:42:23.684213 30065 catalog_manager.cc:5495] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b9214ae009a4eb9ae7d135f4910154a (127.31.59.195). New cstate: current_term: 1 leader_uuid: "8b9214ae009a4eb9ae7d135f4910154a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6df51292f04b02a5b791b349ce0a2c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b9214ae009a4eb9ae7d135f4910154a" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 42573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "975603f9b663482cba3f87b6745f189d" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38283 } health_report { overall_health: UNKNOWN } } }
W20260810 06:42:23.689167 30393 tablet.cc:2369] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:42:23.705587 30643 tablet.cc:2369] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:42:23.718735 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:23.713583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33068 (local address 127.31.59.195:42573)
0810 06:42:23.713793 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:23.713799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:23.713872 (+ 73us) server_negotiation.cc:407] Connection header received
0810 06:42:23.714035 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:23.714038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:23.714116 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:23.714234 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:42:23.714596 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.715278 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.716584 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.716970 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.717062 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:23.717525 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:42:23.717980 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:42:23.718115 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:23.718408 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 06:42:23.718461 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 06:42:23.721509 30890 tablet.cc:2369] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:42:23.727170 30558 raft_consensus.cc:1261] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d [term 1 FOLLOWER]: Refusing update from remote peer 8b9214ae009a4eb9ae7d135f4910154a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:42:23.727195 30311 raft_consensus.cc:1261] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c [term 1 FOLLOWER]: Refusing update from remote peer 8b9214ae009a4eb9ae7d135f4910154a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:42:23.728039 30940 consensus_queue.cc:1035] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [LEADER]: Connected to new peer: Peer: permanent_uuid: "975603f9b663482cba3f87b6745f189d" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:42:23.728369 30934 consensus_queue.cc:1035] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a6df51292f04b02a5b791b349ce0a2c" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:42:23.739521 30945 mvcc.cc:205] Tried to move back new op lower bound from 7316865612696948736 to 7316865612517027840. Current Snapshot: MvccSnapshot[applied={T|T < 7316865612696948736}]
I20260810 06:42:23.739578 30953 mvcc.cc:205] Tried to move back new op lower bound from 7316865612696948736 to 7316865612517027840. Current Snapshot: MvccSnapshot[applied={T|T < 7316865612696948736}]
I20260810 06:42:23.752589 30065 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:42:23.752867 30065 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:42:23.756369 30065 catalog_manager.cc:5782] T 00000000000000000000000000000000 P edb8f7c2118c4508b5fca88e363dfd81: Sending DeleteTablet for 3 replicas of tablet 9aab848994cd4997a9dd4cd8afffbae0
I20260810 06:42:23.757395 30276 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aab848994cd4997a9dd4cd8afffbae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:42:23 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:49700
I20260810 06:42:23.757730 30526 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aab848994cd4997a9dd4cd8afffbae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:42:23 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:50034
I20260810 06:42:23.757792 30759 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aab848994cd4997a9dd4cd8afffbae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:42:23 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:33054
I20260810 06:42:23.759513 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 30170
I20260810 06:42:23.773852 30973 tablet_replica.cc:330] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a: stopping tablet replica
I20260810 06:42:23.774700 30971 tablet_replica.cc:330] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d: stopping tablet replica
I20260810 06:42:23.774883 30970 tablet_replica.cc:330] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c: stopping tablet replica
I20260810 06:42:23.775166 30971 raft_consensus.cc:2229] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:42:23.775319 30970 raft_consensus.cc:2229] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:42:23.775702 30971 raft_consensus.cc:2258] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:42:23.775803 30970 raft_consensus.cc:2258] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:42:23.776433 30973 raft_consensus.cc:2229] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:42:23.777112 30973 raft_consensus.cc:2258] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:42:23.778085 30971 ts_tablet_manager.cc:1905] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:42:23.778157 30970 ts_tablet_manager.cc:1905] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:42:23.779901 30973 ts_tablet_manager.cc:1905] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:42:23.785401 30970 ts_tablet_manager.cc:1918] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:42:23.785571 30970 log.cc:1196] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/9aab848994cd4997a9dd4cd8afffbae0
I20260810 06:42:23.786114 30970 ts_tablet_manager.cc:1939] T 9aab848994cd4997a9dd4cd8afffbae0 P 6a6df51292f04b02a5b791b349ce0a2c: Deleting consensus metadata
I20260810 06:42:23.787411 30973 ts_tablet_manager.cc:1918] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:42:23.787650 30973 log.cc:1196] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/9aab848994cd4997a9dd4cd8afffbae0
I20260810 06:42:23.788013 30038 catalog_manager.cc:4841] TS 6a6df51292f04b02a5b791b349ce0a2c (127.31.59.193:45763): tablet 9aab848994cd4997a9dd4cd8afffbae0 (table test-table [id=5fcf88fd4c214f73b15a3931b4167fdb]) successfully deleted
I20260810 06:42:23.788273 30973 ts_tablet_manager.cc:1939] T 9aab848994cd4997a9dd4cd8afffbae0 P 8b9214ae009a4eb9ae7d135f4910154a: Deleting consensus metadata
Aug 10 06:42:23 dist-test-slave-4tdj krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:42:23.789933 30035 catalog_manager.cc:4841] TS 8b9214ae009a4eb9ae7d135f4910154a (127.31.59.195:42573): tablet 9aab848994cd4997a9dd4cd8afffbae0 (table test-table [id=5fcf88fd4c214f73b15a3931b4167fdb]) successfully deleted
I20260810 06:42:23.789979 30971 ts_tablet_manager.cc:1918] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:42:23.790172 30971 log.cc:1196] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/9aab848994cd4997a9dd4cd8afffbae0
I20260810 06:42:23.790608 30971 ts_tablet_manager.cc:1939] T 9aab848994cd4997a9dd4cd8afffbae0 P 975603f9b663482cba3f87b6745f189d: Deleting consensus metadata
I20260810 06:42:23.792186 30036 catalog_manager.cc:4841] TS 975603f9b663482cba3f87b6745f189d (127.31.59.194:38283): tablet 9aab848994cd4997a9dd4cd8afffbae0 (table test-table [id=5fcf88fd4c214f73b15a3931b4167fdb]) successfully deleted
I20260810 06:42:23.796954 30913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:23.760425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49712 (local address 127.31.59.193:45763)
0810 06:42:23.760602 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:23.760609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:23.774019 (+ 13410us) server_negotiation.cc:407] Connection header received
0810 06:42:23.774228 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:23.774234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:23.774351 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:23.774447 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:23.776559 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.777692 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.779565 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.780236 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.780336 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:23.790784 (+ 10448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:23.790799 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:23.790816 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:23.790857 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:23.794025 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:23.794911 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:23.794917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:23.794919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:23.795045 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:23.795440 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:23.795443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:23.795445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:23.795825 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:23.796047 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:23.796420 (+ 373us) server_negotiation.cc:299] Negotiation successful
0810 06:42:23.796622 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:42:23.797647 30382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:23.901187 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 30170
I20260810 06:42:23.929770 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 30404
Aug 10 06:42:23 dist-test-slave-4tdj krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.194@KRBTEST.COM
I20260810 06:42:23.946223 30917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:23.930514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50044 (local address 127.31.59.194:38283)
0810 06:42:23.930721 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:23.930726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:23.930769 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:42:23.930960 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:23.930964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:23.931026 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:23.931108 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:23.932786 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.933504 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.935341 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:23.935736 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:23.935826 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:23.940752 (+ 4926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:23.940767 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:23.940784 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:23.940818 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:23.943524 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:23.944307 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:23.944314 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:23.944316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:23.944433 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:23.944831 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:23.944835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:23.944837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:23.945210 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:23.945435 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:23.945797 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 06:42:23.945957 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260810 06:42:23.946882 30635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:24.038998 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 30404
I20260810 06:42:24.062456 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 30649
Aug 10 06:42:24 dist-test-slave-4tdj krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.195@KRBTEST.COM
I20260810 06:42:24.080370 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:24.063423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33072 (local address 127.31.59.195:42573)
0810 06:42:24.063589 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:24.063593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:24.063614 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:42:24.063717 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:24.063721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:24.063795 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:24.063871 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:24.065787 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:24.066839 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:24.068646 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:24.069102 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:24.069203 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:24.074619 (+ 5416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:24.074633 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:24.074656 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:24.074692 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:24.077602 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:24.078215 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:24.078223 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:24.078229 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:24.078436 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:24.078817 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:24.078822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:24.078826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:24.079206 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:24.079504 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:24.079783 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 06:42:24.080001 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:42:24.081166 30877 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:24.202196 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 30649
I20260810 06:42:24.228319 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 30004
I20260810 06:42:24.229367 30115 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:24.337893 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 30004
2026-08-10T06:42:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4442 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T06:42:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:42:24Z Disabled control of system clock
I20260810 06:42:24.424160 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:36569
--webserver_interface=127.31.59.254
--webserver_port=0
--builtin_ntp_servers=127.31.59.212:37195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:36569
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:42:24.747922 30991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:24.748205 30991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:24.748301 30991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:24.755504 30991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:42:24.755621 30991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:24.755676 30991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:24.755739 30991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:42:24.755789 30991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:42:24.764719 30991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:37195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:36569
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:36569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:24.765864 30991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:24.767643 30991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:24.778940 31003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:24.779003 31001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:24.779027 31000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:24.779844 30991 server_base.cc:1034] running on GCE node
I20260810 06:42:24.780575 30991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:24.781873 30991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:24.783085 30991 hybrid_clock.cc:648] HybridClock initialized: now 1786344144783028 us; error 45 us; skew 500 ppm
I20260810 06:42:24.783437 30991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:42:24.785740 30991 webserver.cc:460] Webserver started at http://127.31.59.254:38875/ using document root <none> and password file <none>
I20260810 06:42:24.786417 30991 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:24.786513 30991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:24.786821 30991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:24.789520 30991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "5b65071d0c60422d8267306639a1ffa0"
format_stamp: "Formatted at 2026-08-10 06:42:24 on dist-test-slave-4tdj"
server_key: "e60363447a2f88a52257b512a04e243e"
server_key_iv: "2bc12ee70828aa74849287d6a085c8f4"
server_key_version: "encryptionkey@0"
I20260810 06:42:24.790321 30991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "5b65071d0c60422d8267306639a1ffa0"
format_stamp: "Formatted at 2026-08-10 06:42:24 on dist-test-slave-4tdj"
server_key: "e60363447a2f88a52257b512a04e243e"
server_key_iv: "2bc12ee70828aa74849287d6a085c8f4"
server_key_version: "encryptionkey@0"
I20260810 06:42:24.795222 30991 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260810 06:42:24.798422 31011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:24.799887 30991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:42:24.800040 30991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "5b65071d0c60422d8267306639a1ffa0"
format_stamp: "Formatted at 2026-08-10 06:42:24 on dist-test-slave-4tdj"
server_key: "e60363447a2f88a52257b512a04e243e"
server_key_iv: "2bc12ee70828aa74849287d6a085c8f4"
server_key_version: "encryptionkey@0"
I20260810 06:42:24.800230 30991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:24.813463 30991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:24.814172 30991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:24.814455 30991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:24.830111 30991 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:36569
I20260810 06:42:24.830143 31099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:36569 every 8 connection(s)
I20260810 06:42:24.831370 30991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:42:24.834991 31100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:24.835876 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 30991
I20260810 06:42:24.836071 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:42:24.836623 31983 external_mini_cluster.cc:1428] Setting key cc29496e5005a28f087d9f388a640e14
I20260810 06:42:24.849145 31100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0: Bootstrap starting.
I20260810 06:42:24.851400 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:24.838503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42268 (local address 127.31.59.254:36569)
0810 06:42:24.839547 (+ 1044us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:24.839561 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:24.839614 (+ 53us) server_negotiation.cc:407] Connection header received
0810 06:42:24.841686 (+ 2072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:24.841717 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:24.842287 (+ 570us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:24.842733 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:24.843233 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:24.845006 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:24.846462 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:24.847454 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:24.847616 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:24.848638 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:24.848648 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:24.848668 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:24.849414 (+ 746us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:24.849508 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:24.849534 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:24.849643 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:24.849847 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 06:42:24.850141 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":342,"threads_started":1}
I20260810 06:42:24.853001 31100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:24.854084 31100 log.cc:824] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:24.856505 31100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0: No bootstrap required, opened a new log
I20260810 06:42:24.863324 31100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b65071d0c60422d8267306639a1ffa0" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36569 } }
I20260810 06:42:24.863647 31100 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:24.863745 31100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b65071d0c60422d8267306639a1ffa0, State: Initialized, Role: FOLLOWER
I20260810 06:42:24.864272 31100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b65071d0c60422d8267306639a1ffa0" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36569 } }
I20260810 06:42:24.864485 31100 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:42:24.864616 31100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:42:24.864754 31100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:24.866819 31100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b65071d0c60422d8267306639a1ffa0" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36569 } }
I20260810 06:42:24.867363 31100 leader_election.cc:302] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b65071d0c60422d8267306639a1ffa0; no voters:
I20260810 06:42:24.867877 31100 leader_election.cc:288] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:42:24.868005 31111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:24.868716 31111 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0 [term 1 LEADER]: Becoming Leader. State: Replica: 5b65071d0c60422d8267306639a1ffa0, State: Running, Role: LEADER
I20260810 06:42:24.869163 31111 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b65071d0c60422d8267306639a1ffa0" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36569 } }
I20260810 06:42:24.869691 31100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:42:24.871822 31115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b65071d0c60422d8267306639a1ffa0. Latest consensus state: current_term: 1 leader_uuid: "5b65071d0c60422d8267306639a1ffa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b65071d0c60422d8267306639a1ffa0" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36569 } } }
I20260810 06:42:24.872213 31115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:24.871927 31112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b65071d0c60422d8267306639a1ffa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b65071d0c60422d8267306639a1ffa0" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 36569 } } }
I20260810 06:42:24.872450 31112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:24.877686 31124 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:42:24.882480 31124 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:42:24.890877 31124 catalog_manager.cc:1347] Generated new cluster ID: f5485242affa40b4982bc6d93d8d1d68
I20260810 06:42:24.890977 31124 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:42:24.920584 31124 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:42:24.921845 31124 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:42:24.933038 31124 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0: Generated new TSK 0
I20260810 06:42:24.933866 31124 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:42:24.945996 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36569
--builtin_ntp_servers=127.31.59.212:37195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:42:25.313261 31146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:25.313532 31146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:25.313663 31146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:25.320855 31146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:25.320959 31146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:25.321110 31146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:42:25.329916 31146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:37195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:25.331154 31146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:25.332875 31146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:25.345278 31158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:25.345230 31160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:25.345435 31163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:25.346190 31146 server_base.cc:1034] running on GCE node
I20260810 06:42:25.346778 31146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:25.347685 31146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:25.348881 31146 hybrid_clock.cc:648] HybridClock initialized: now 1786344145348827 us; error 41 us; skew 500 ppm
I20260810 06:42:25.349154 31146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:42:25.351279 31146 webserver.cc:460] Webserver started at http://127.31.59.193:38481/ using document root <none> and password file <none>
I20260810 06:42:25.351894 31146 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:25.351980 31146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:25.352234 31146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:25.355000 31146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "4cd6516fe1ee41dfb259c5621a6b7caa"
format_stamp: "Formatted at 2026-08-10 06:42:25 on dist-test-slave-4tdj"
server_key: "f104454a900c32352ba5ef4a22d580b6"
server_key_iv: "9404d27c1caa2c2c5a16233ac9933406"
server_key_version: "encryptionkey@0"
I20260810 06:42:25.355759 31146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "4cd6516fe1ee41dfb259c5621a6b7caa"
format_stamp: "Formatted at 2026-08-10 06:42:25 on dist-test-slave-4tdj"
server_key: "f104454a900c32352ba5ef4a22d580b6"
server_key_iv: "9404d27c1caa2c2c5a16233ac9933406"
server_key_version: "encryptionkey@0"
I20260810 06:42:25.360489 31146 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 06:42:25.363690 31169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:25.364871 31146 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:42:25.365016 31146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "4cd6516fe1ee41dfb259c5621a6b7caa"
format_stamp: "Formatted at 2026-08-10 06:42:25 on dist-test-slave-4tdj"
server_key: "f104454a900c32352ba5ef4a22d580b6"
server_key_iv: "9404d27c1caa2c2c5a16233ac9933406"
server_key_version: "encryptionkey@0"
I20260810 06:42:25.365203 31146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:25.379133 31146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:25.379992 31146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:25.380307 31146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:25.381237 31146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:42:25.382607 31146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:42:25.382710 31146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:25.382810 31146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:42:25.382882 31146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:25.414750 31146 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:33719
I20260810 06:42:25.414816 31373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:33719 every 8 connection(s)
I20260810 06:42:25.416004 31146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:42:25.418699 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 31146
I20260810 06:42:25.418867 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:42:25.419410 31983 external_mini_cluster.cc:1428] Setting key db2e6f60ba26181f018fc56008ffaa9c
I20260810 06:42:25.422178 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36569
--builtin_ntp_servers=127.31.59.212:37195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:42:25.428285 31378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:25.418442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:51257 (local address 127.31.59.254:36569)
0810 06:42:25.418901 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:25.418907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:25.422107 (+ 3200us) server_negotiation.cc:407] Connection header received
0810 06:42:25.422163 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:25.422166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:25.422228 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:25.422364 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:25.423460 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:25.424195 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:25.425698 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:25.426102 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:25.426213 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:25.427486 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:25.427491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:25.427494 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:25.427512 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:25.427552 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:25.427563 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:25.427637 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:25.427840 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 06:42:25.427934 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":232,"threads_started":1}
I20260810 06:42:25.429189 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:25.419114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36569 (local address 127.31.59.193:51257)
0810 06:42:25.420091 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:25.420147 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:25.421836 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:25.422662 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:25.422670 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:25.422753 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:42:25.423298 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:25.423315 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:25.424344 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:25.424347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:25.425561 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:25.425573 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:25.426335 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:25.426343 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:25.426577 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:25.427199 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:42:25.427202 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:42:25.427239 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:42:25.427311 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:42:25.427651 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:42:25.427662 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:42:25.427670 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:42:25.427816 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 06:42:25.428180 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":320,"threads_started":1}
I20260810 06:42:25.430796 31374 heartbeater.cc:347] Connected to a master server at 127.31.59.254:36569
I20260810 06:42:25.431068 31374 heartbeater.cc:464] Registering TS with master...
I20260810 06:42:25.431783 31374 heartbeater.cc:511] Master 127.31.59.254:36569 requested a full tablet report, sending...
I20260810 06:42:25.433815 31043 ts_manager.cc:194] Registered new tserver with Master: 4cd6516fe1ee41dfb259c5621a6b7caa (127.31.59.193:33719)
I20260810 06:42:25.435330 31043 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.193:51257
W20260810 06:42:25.747022 31381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:25.747291 31381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:25.747387 31381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:25.754526 31381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:25.754627 31381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:25.754739 31381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:42:25.763615 31381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:37195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:25.764796 31381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:25.766593 31381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:25.779312 31387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:25.779534 31393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:25.779754 31396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:25.780300 31381 server_base.cc:1034] running on GCE node
I20260810 06:42:25.780862 31381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:25.781819 31381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:25.783037 31381 hybrid_clock.cc:648] HybridClock initialized: now 1786344145782980 us; error 38 us; skew 500 ppm
I20260810 06:42:25.783337 31381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:42:25.785495 31381 webserver.cc:460] Webserver started at http://127.31.59.194:46437/ using document root <none> and password file <none>
I20260810 06:42:25.786141 31381 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:25.786244 31381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:25.786589 31381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:25.789322 31381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "cd8d1c9d9310417b9242bb3e16e4c556"
format_stamp: "Formatted at 2026-08-10 06:42:25 on dist-test-slave-4tdj"
server_key: "5bba65e27a4661189e789f3ce5e5ec8b"
server_key_iv: "11d8f78bba32269b9303444b572aae5c"
server_key_version: "encryptionkey@0"
I20260810 06:42:25.790091 31381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "cd8d1c9d9310417b9242bb3e16e4c556"
format_stamp: "Formatted at 2026-08-10 06:42:25 on dist-test-slave-4tdj"
server_key: "5bba65e27a4661189e789f3ce5e5ec8b"
server_key_iv: "11d8f78bba32269b9303444b572aae5c"
server_key_version: "encryptionkey@0"
I20260810 06:42:25.795070 31381 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 06:42:25.798447 31405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:25.799729 31381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:42:25.799863 31381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "cd8d1c9d9310417b9242bb3e16e4c556"
format_stamp: "Formatted at 2026-08-10 06:42:25 on dist-test-slave-4tdj"
server_key: "5bba65e27a4661189e789f3ce5e5ec8b"
server_key_iv: "11d8f78bba32269b9303444b572aae5c"
server_key_version: "encryptionkey@0"
I20260810 06:42:25.800036 31381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:25.821430 31381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:25.822175 31381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:25.822477 31381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:25.823391 31381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:42:25.824748 31381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:42:25.824833 31381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:25.824910 31381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:42:25.824962 31381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:25.855594 31381 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:42155
I20260810 06:42:25.855677 31612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:42155 every 8 connection(s)
I20260810 06:42:25.856806 31381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:42:25.860780 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 31381
I20260810 06:42:25.860945 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:42:25.861366 31983 external_mini_cluster.cc:1428] Setting key 71904fc8506c4b32b452b516cfcfc6a1
I20260810 06:42:25.864269 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36569
--builtin_ntp_servers=127.31.59.212:37195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:42:25.869529 31378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:25.859170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:47739 (local address 127.31.59.254:36569)
0810 06:42:25.859331 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:25.859336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:25.860698 (+ 1362us) server_negotiation.cc:407] Connection header received
0810 06:42:25.862696 (+ 1998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:25.862700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:25.862764 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:25.862863 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:25.864457 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:25.865436 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:25.866998 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:25.867531 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:25.867649 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:25.868821 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:25.868827 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:25.868831 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:25.868850 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:25.868887 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:25.868901 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:25.868986 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:25.869165 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 06:42:25.869256 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:42:25.870644 31616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:25.859682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36569 (local address 127.31.59.194:47739)
0810 06:42:25.860539 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:25.860574 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:25.862396 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:25.863564 (+ 1168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:25.863577 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:25.863694 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:42:25.864280 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:25.864296 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:25.865600 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:25.865604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:25.866862 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:25.866874 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:25.867706 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:25.867711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:25.867898 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:25.868530 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:42:25.868533 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:42:25.868570 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:42:25.868646 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:42:25.868980 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:42:25.868986 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:42:25.868990 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:42:25.869135 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 06:42:25.869529 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":261,"threads_started":1}
I20260810 06:42:25.871894 31613 heartbeater.cc:347] Connected to a master server at 127.31.59.254:36569
I20260810 06:42:25.872208 31613 heartbeater.cc:464] Registering TS with master...
I20260810 06:42:25.873018 31613 heartbeater.cc:511] Master 127.31.59.254:36569 requested a full tablet report, sending...
I20260810 06:42:25.874507 31043 ts_manager.cc:194] Registered new tserver with Master: cd8d1c9d9310417b9242bb3e16e4c556 (127.31.59.194:42155)
I20260810 06:42:25.875283 31043 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.194:47739
W20260810 06:42:26.192986 31618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:26.193259 31618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:26.193389 31618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:26.200449 31618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:26.200585 31618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:26.200732 31618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:42:26.209673 31618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:37195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:36569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:26.210956 31618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:26.212762 31618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:26.225008 31628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:26.225071 31629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:26.225034 31633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:26.225768 31618 server_base.cc:1034] running on GCE node
I20260810 06:42:26.226413 31618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:26.227304 31618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:26.228498 31618 hybrid_clock.cc:648] HybridClock initialized: now 1786344146228454 us; error 37 us; skew 500 ppm
I20260810 06:42:26.228773 31618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:42:26.230861 31618 webserver.cc:460] Webserver started at http://127.31.59.195:32843/ using document root <none> and password file <none>
I20260810 06:42:26.231482 31618 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:26.231567 31618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:26.231824 31618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:26.234599 31618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "5f61d869461845e09b13896a0bbc4add"
format_stamp: "Formatted at 2026-08-10 06:42:26 on dist-test-slave-4tdj"
server_key: "d59fda39dce8d703f4b5725a2e824d2f"
server_key_iv: "c706424e643590b62fbe9c9eb25c1a0a"
server_key_version: "encryptionkey@0"
I20260810 06:42:26.235409 31618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "5f61d869461845e09b13896a0bbc4add"
format_stamp: "Formatted at 2026-08-10 06:42:26 on dist-test-slave-4tdj"
server_key: "d59fda39dce8d703f4b5725a2e824d2f"
server_key_iv: "c706424e643590b62fbe9c9eb25c1a0a"
server_key_version: "encryptionkey@0"
I20260810 06:42:26.240537 31618 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:42:26.243948 31642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:26.245272 31618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:42:26.245404 31618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "5f61d869461845e09b13896a0bbc4add"
format_stamp: "Formatted at 2026-08-10 06:42:26 on dist-test-slave-4tdj"
server_key: "d59fda39dce8d703f4b5725a2e824d2f"
server_key_iv: "c706424e643590b62fbe9c9eb25c1a0a"
server_key_version: "encryptionkey@0"
I20260810 06:42:26.245570 31618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:26.268021 31618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:26.268872 31618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:26.269133 31618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:26.270167 31618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:42:26.271572 31618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:42:26.271656 31618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:26.271757 31618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:42:26.271828 31618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:26.301368 31618 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:35789
I20260810 06:42:26.301441 31856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:35789 every 8 connection(s)
I20260810 06:42:26.302732 31618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:42:26.303201 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 31618
I20260810 06:42:26.303397 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:42:26.303963 31983 external_mini_cluster.cc:1428] Setting key ffb5f013f6c2fd29de9f587004a86705
I20260810 06:42:26.314915 31378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.305505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:43837 (local address 127.31.59.254:36569)
0810 06:42:26.305662 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:26.305666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:26.307258 (+ 1592us) server_negotiation.cc:407] Connection header received
0810 06:42:26.308898 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:26.308902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:26.308968 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:26.309049 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:26.309985 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.310961 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.312533 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.312945 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.313035 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:26.314263 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:26.314268 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:26.314271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:26.314286 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:26.314351 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:26.314366 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:26.314435 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:26.314607 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 06:42:26.314680 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:42:26.315860 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.306132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:36569 (local address 127.31.59.195:43837)
0810 06:42:26.307119 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:26.307153 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:26.308673 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:26.309205 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:26.309212 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:26.309294 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:42:26.309856 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:26.309872 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.311115 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.311120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:26.312402 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:26.312417 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.313118 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.313123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:26.313334 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:26.313997 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:42:26.314000 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:42:26.314038 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:42:26.314120 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:42:26.314436 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:42:26.314444 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:42:26.314449 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:42:26.314599 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 06:42:26.314873 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":310,"threads_started":1}
I20260810 06:42:26.317232 31858 heartbeater.cc:347] Connected to a master server at 127.31.59.254:36569
I20260810 06:42:26.317548 31858 heartbeater.cc:464] Registering TS with master...
I20260810 06:42:26.318346 31858 heartbeater.cc:511] Master 127.31.59.254:36569 requested a full tablet report, sending...
I20260810 06:42:26.319741 31043 ts_manager.cc:194] Registered new tserver with Master: 5f61d869461845e09b13896a0bbc4add (127.31.59.195:35789)
I20260810 06:42:26.320519 31043 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.195:43837
I20260810 06:42:26.331072 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:42:26.340281 31378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.334929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42276 (local address 127.31.59.254:36569)
0810 06:42:26.335088 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:26.335093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:26.335532 (+ 439us) server_negotiation.cc:407] Connection header received
0810 06:42:26.335676 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:26.335679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:26.335749 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:26.335823 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:26.336214 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.336872 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.338125 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.338562 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.338653 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:26.339645 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:26.339650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:26.339653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:26.339669 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:26.339699 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:26.339710 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:26.339814 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:26.340031 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 06:42:26.340072 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:42:26.343961 31043 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:42:26.350651 31043 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:42:26.371986 31881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.362044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:42155 (local address 127.0.0.1:37396)
0810 06:42:26.362691 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:26.362741 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:26.362912 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:26.363897 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:26.363906 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:26.364829 (+ 923us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:26.365252 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:26.365266 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.366974 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.366979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:26.369128 (+ 2149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:26.369141 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.371060 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.371066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:26.371248 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:26.371299 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 06:42:26.371421 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 06:42:26.371614 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":69,"thread_start_us":281,"threads_started":1}
I20260810 06:42:26.372777 31885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.363653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:35789 (local address 127.0.0.1:58614)
0810 06:42:26.364152 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:26.364168 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:26.364275 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:26.365532 (+ 1257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:26.365536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:26.365562 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:26.365903 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:26.365917 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.367644 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.367653 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:26.370004 (+ 2351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:26.370019 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.371995 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.372002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:26.372193 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:26.372227 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 06:42:26.372299 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 06:42:26.372423 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":42,"thread_start_us":313,"threads_started":1}
I20260810 06:42:26.373003 31882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.362438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37396 (local address 127.31.59.194:42155)
0810 06:42:26.362966 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:26.362982 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:26.363013 (+ 31us) server_negotiation.cc:407] Connection header received
0810 06:42:26.363108 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:26.363116 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:26.363347 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:26.363542 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:26.365404 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.366787 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.369306 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.370537 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.370665 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:26.370934 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:26.372384 (+ 1450us) server_negotiation.cc:299] Negotiation successful
0810 06:42:26.372570 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":231,"threads_started":1}
I20260810 06:42:26.373112 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.363061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:33719 (local address 127.0.0.1:58734)
0810 06:42:26.363574 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:26.363590 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:26.363697 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:26.365872 (+ 2175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:26.365876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:26.365904 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:26.366281 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:26.366319 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.368093 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.368098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:26.370262 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:26.370274 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.372389 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.372396 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:26.372562 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:26.372594 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:42:26.372656 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 06:42:26.372770 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":5,"thread_start_us":241,"threads_started":1}
I20260810 06:42:26.373181 31886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.364190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58614 (local address 127.31.59.195:35789)
0810 06:42:26.364943 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:26.364961 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:26.364993 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:42:26.365097 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:26.365104 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:26.365341 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:26.365529 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:26.366143 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.367478 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.370196 (+ 2718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.371783 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.371909 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:26.372442 (+ 533us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:26.372502 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 06:42:26.372636 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":421,"threads_started":1}
I20260810 06:42:26.373970 31884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.363358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58734 (local address 127.31.59.193:33719)
0810 06:42:26.365284 (+ 1926us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:26.365291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:26.365318 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:42:26.365419 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:26.365426 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:26.365632 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:26.365868 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:26.366463 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.367922 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.370508 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.372108 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.373325 (+ 1217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:26.373536 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:26.373595 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 06:42:26.373705 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1761,"thread_start_us":1651,"threads_started":1}
I20260810 06:42:26.376880 31736 tablet_service.cc:1467] Processing CreateTablet for tablet 94b3b93d6d08466ca645a6e4d9d9f476 (DEFAULT_TABLE table=test-table [id=d13402c125f4480abf72b4bba14606b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:42:26.376861 31476 tablet_service.cc:1467] Processing CreateTablet for tablet 94b3b93d6d08466ca645a6e4d9d9f476 (DEFAULT_TABLE table=test-table [id=d13402c125f4480abf72b4bba14606b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:42:26.377187 31249 tablet_service.cc:1467] Processing CreateTablet for tablet 94b3b93d6d08466ca645a6e4d9d9f476 (DEFAULT_TABLE table=test-table [id=d13402c125f4480abf72b4bba14606b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:42:26.378324 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94b3b93d6d08466ca645a6e4d9d9f476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:26.378321 31476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94b3b93d6d08466ca645a6e4d9d9f476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:26.378579 31249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94b3b93d6d08466ca645a6e4d9d9f476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:26.390965 31887 tablet_bootstrap.cc:492] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa: Bootstrap starting.
I20260810 06:42:26.394320 31887 tablet_bootstrap.cc:654] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:26.395346 31887 log.cc:824] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:26.397549 31888 tablet_bootstrap.cc:492] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556: Bootstrap starting.
I20260810 06:42:26.398254 31887 tablet_bootstrap.cc:492] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa: No bootstrap required, opened a new log
I20260810 06:42:26.398528 31887 ts_tablet_manager.cc:1397] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260810 06:42:26.398514 31890 tablet_bootstrap.cc:492] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add: Bootstrap starting.
I20260810 06:42:26.400882 31888 tablet_bootstrap.cc:654] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:26.401815 31890 tablet_bootstrap.cc:654] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:26.402112 31888 log.cc:824] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:26.403056 31890 log.cc:824] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:26.405624 31888 tablet_bootstrap.cc:492] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556: No bootstrap required, opened a new log
I20260810 06:42:26.405915 31888 ts_tablet_manager.cc:1397] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 06:42:26.406049 31887 raft_consensus.cc:348] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8d1c9d9310417b9242bb3e16e4c556" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42155 } } peers { permanent_uuid: "4cd6516fe1ee41dfb259c5621a6b7caa" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33719 } } peers { permanent_uuid: "5f61d869461845e09b13896a0bbc4add" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 35789 } }
I20260810 06:42:26.406417 31887 raft_consensus.cc:374] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:26.406523 31887 raft_consensus.cc:729] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd6516fe1ee41dfb259c5621a6b7caa, State: Initialized, Role: FOLLOWER
I20260810 06:42:26.407051 31887 consensus_queue.cc:260] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8d1c9d9310417b9242bb3e16e4c556" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42155 } } peers { permanent_uuid: "4cd6516fe1ee41dfb259c5621a6b7caa" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33719 } } peers { permanent_uuid: "5f61d869461845e09b13896a0bbc4add" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 35789 } }
I20260810 06:42:26.408061 31374 heartbeater.cc:503] Master 127.31.59.254:36569 was elected leader, sending a full tablet report...
I20260810 06:42:26.408548 31887 ts_tablet_manager.cc:1428] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260810 06:42:26.413030 31890 tablet_bootstrap.cc:492] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add: No bootstrap required, opened a new log
I20260810 06:42:26.413374 31890 ts_tablet_manager.cc:1397] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.008s
I20260810 06:42:26.414917 31888 raft_consensus.cc:348] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8d1c9d9310417b9242bb3e16e4c556" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42155 } } peers { permanent_uuid: "4cd6516fe1ee41dfb259c5621a6b7caa" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33719 } } peers { permanent_uuid: "5f61d869461845e09b13896a0bbc4add" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 35789 } }
I20260810 06:42:26.415350 31888 raft_consensus.cc:374] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:26.415444 31888 raft_consensus.cc:729] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd8d1c9d9310417b9242bb3e16e4c556, State: Initialized, Role: FOLLOWER
I20260810 06:42:26.416046 31888 consensus_queue.cc:260] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8d1c9d9310417b9242bb3e16e4c556" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42155 } } peers { permanent_uuid: "4cd6516fe1ee41dfb259c5621a6b7caa" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33719 } } peers { permanent_uuid: "5f61d869461845e09b13896a0bbc4add" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 35789 } }
I20260810 06:42:26.417086 31613 heartbeater.cc:503] Master 127.31.59.254:36569 was elected leader, sending a full tablet report...
I20260810 06:42:26.417671 31888 ts_tablet_manager.cc:1428] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
W20260810 06:42:26.418040 31375 tablet.cc:2369] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:42:26.423285 31890 raft_consensus.cc:348] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8d1c9d9310417b9242bb3e16e4c556" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42155 } } peers { permanent_uuid: "4cd6516fe1ee41dfb259c5621a6b7caa" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33719 } } peers { permanent_uuid: "5f61d869461845e09b13896a0bbc4add" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 35789 } }
I20260810 06:42:26.423599 31890 raft_consensus.cc:374] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:26.423686 31890 raft_consensus.cc:729] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f61d869461845e09b13896a0bbc4add, State: Initialized, Role: FOLLOWER
I20260810 06:42:26.424202 31890 consensus_queue.cc:260] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8d1c9d9310417b9242bb3e16e4c556" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42155 } } peers { permanent_uuid: "4cd6516fe1ee41dfb259c5621a6b7caa" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33719 } } peers { permanent_uuid: "5f61d869461845e09b13896a0bbc4add" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 35789 } }
I20260810 06:42:26.425077 31895 raft_consensus.cc:482] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:42:26.425379 31895 raft_consensus.cc:504] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8d1c9d9310417b9242bb3e16e4c556" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42155 } } peers { permanent_uuid: "4cd6516fe1ee41dfb259c5621a6b7caa" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33719 } } peers { permanent_uuid: "5f61d869461845e09b13896a0bbc4add" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 35789 } }
I20260810 06:42:26.425791 31890 ts_tablet_manager.cc:1428] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add: Time spent starting tablet: real 0.012s user 0.006s sys 0.003s
I20260810 06:42:26.426726 31858 heartbeater.cc:503] Master 127.31.59.254:36569 was elected leader, sending a full tablet report...
I20260810 06:42:26.427278 31895 leader_election.cc:288] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd8d1c9d9310417b9242bb3e16e4c556 (127.31.59.194:42155), 5f61d869461845e09b13896a0bbc4add (127.31.59.195:35789)
I20260810 06:42:26.435985 31886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.428234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:52861 (local address 127.31.59.195:35789)
0810 06:42:26.428392 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:26.428398 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:26.428801 (+ 403us) server_negotiation.cc:407] Connection header received
0810 06:42:26.428951 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:26.428956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:26.429022 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:26.429118 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:26.429727 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.430732 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.433699 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.434920 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.435055 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:26.435123 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:26.435634 (+ 511us) server_negotiation.cc:299] Negotiation successful
0810 06:42:26.435756 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 06:42:26.436010 31882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.427472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:38203 (local address 127.31.59.194:42155)
0810 06:42:26.427645 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:26.427651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:26.428166 (+ 515us) server_negotiation.cc:407] Connection header received
0810 06:42:26.428617 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:26.428622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:26.428706 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:26.428775 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:26.429548 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.430644 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.433683 (+ 3039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.434898 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.435067 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:26.435124 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:26.435634 (+ 510us) server_negotiation.cc:299] Negotiation successful
0810 06:42:26.435756 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 06:42:26.436230 31898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.427413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:42155 (local address 127.31.59.193:38203)
0810 06:42:26.428065 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:26.428084 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:26.428434 (+ 350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:26.428972 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:26.428977 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:26.429014 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:26.429402 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:26.429414 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.430822 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.430829 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:26.433462 (+ 2633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:26.433480 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.435132 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.435138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:26.435428 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:26.435511 (+ 83us) client_negotiation.cc:769] Sending connection context
0810 06:42:26.435608 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 06:42:26.435812 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":269,"thread_start_us":371,"threads_started":1}
I20260810 06:42:26.436245 31901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.428174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:35789 (local address 127.31.59.193:52861)
0810 06:42:26.428709 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:26.428723 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:26.428836 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:26.429158 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:26.429162 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:26.429187 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:26.429593 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:26.429604 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.430854 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.430859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:26.433485 (+ 2626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:26.433496 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.435108 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.435119 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:26.435429 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:26.435507 (+ 78us) client_negotiation.cc:769] Sending connection context
0810 06:42:26.435608 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 06:42:26.435812 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":239,"thread_start_us":269,"threads_started":1}
I20260810 06:42:26.437261 31766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94b3b93d6d08466ca645a6e4d9d9f476" candidate_uuid: "4cd6516fe1ee41dfb259c5621a6b7caa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f61d869461845e09b13896a0bbc4add" is_pre_election: true
I20260810 06:42:26.437286 31524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94b3b93d6d08466ca645a6e4d9d9f476" candidate_uuid: "4cd6516fe1ee41dfb259c5621a6b7caa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd8d1c9d9310417b9242bb3e16e4c556" is_pre_election: true
I20260810 06:42:26.437711 31766 raft_consensus.cc:2454] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd6516fe1ee41dfb259c5621a6b7caa in term 0.
I20260810 06:42:26.437716 31524 raft_consensus.cc:2454] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd6516fe1ee41dfb259c5621a6b7caa in term 0.
I20260810 06:42:26.438277 31177 leader_election.cc:302] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd6516fe1ee41dfb259c5621a6b7caa, cd8d1c9d9310417b9242bb3e16e4c556; no voters:
I20260810 06:42:26.438683 31895 raft_consensus.cc:2781] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:42:26.438812 31895 raft_consensus.cc:482] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:42:26.438917 31895 raft_consensus.cc:3037] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:26.441072 31895 raft_consensus.cc:504] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8d1c9d9310417b9242bb3e16e4c556" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42155 } } peers { permanent_uuid: "4cd6516fe1ee41dfb259c5621a6b7caa" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33719 } } peers { permanent_uuid: "5f61d869461845e09b13896a0bbc4add" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 35789 } }
I20260810 06:42:26.441646 31895 leader_election.cc:288] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [CANDIDATE]: Term 1 election: Requested vote from peers cd8d1c9d9310417b9242bb3e16e4c556 (127.31.59.194:42155), 5f61d869461845e09b13896a0bbc4add (127.31.59.195:35789)
I20260810 06:42:26.442212 31766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94b3b93d6d08466ca645a6e4d9d9f476" candidate_uuid: "4cd6516fe1ee41dfb259c5621a6b7caa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f61d869461845e09b13896a0bbc4add"
I20260810 06:42:26.442204 31524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94b3b93d6d08466ca645a6e4d9d9f476" candidate_uuid: "4cd6516fe1ee41dfb259c5621a6b7caa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd8d1c9d9310417b9242bb3e16e4c556"
I20260810 06:42:26.442411 31524 raft_consensus.cc:3037] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:26.442476 31766 raft_consensus.cc:3037] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:26.444660 31766 raft_consensus.cc:2454] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd6516fe1ee41dfb259c5621a6b7caa in term 1.
I20260810 06:42:26.444674 31524 raft_consensus.cc:2454] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd6516fe1ee41dfb259c5621a6b7caa in term 1.
I20260810 06:42:26.444978 31177 leader_election.cc:302] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd6516fe1ee41dfb259c5621a6b7caa, cd8d1c9d9310417b9242bb3e16e4c556; no voters:
I20260810 06:42:26.445299 31895 raft_consensus.cc:2781] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:26.445732 31895 raft_consensus.cc:686] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [term 1 LEADER]: Becoming Leader. State: Replica: 4cd6516fe1ee41dfb259c5621a6b7caa, State: Running, Role: LEADER
I20260810 06:42:26.446237 31895 consensus_queue.cc:237] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8d1c9d9310417b9242bb3e16e4c556" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42155 } } peers { permanent_uuid: "4cd6516fe1ee41dfb259c5621a6b7caa" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33719 } } peers { permanent_uuid: "5f61d869461845e09b13896a0bbc4add" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 35789 } }
I20260810 06:42:26.450371 31043 catalog_manager.cc:5495] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cd6516fe1ee41dfb259c5621a6b7caa (127.31.59.193). New cstate: current_term: 1 leader_uuid: "4cd6516fe1ee41dfb259c5621a6b7caa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8d1c9d9310417b9242bb3e16e4c556" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd6516fe1ee41dfb259c5621a6b7caa" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f61d869461845e09b13896a0bbc4add" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 35789 } health_report { overall_health: UNKNOWN } } }
I20260810 06:42:26.473673 31884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.468297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58750 (local address 127.31.59.193:33719)
0810 06:42:26.468498 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:26.468502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:26.468519 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:42:26.468697 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:26.468700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:26.468792 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:26.468910 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:42:26.469283 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.470039 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.471682 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.472096 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.472191 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:26.472685 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:42:26.473136 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:42:26.473260 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:26.473426 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 06:42:26.473461 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:42:26.481639 31524 raft_consensus.cc:1261] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556 [term 1 FOLLOWER]: Refusing update from remote peer 4cd6516fe1ee41dfb259c5621a6b7caa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:42:26.481647 31766 raft_consensus.cc:1261] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add [term 1 FOLLOWER]: Refusing update from remote peer 4cd6516fe1ee41dfb259c5621a6b7caa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:42:26.482555 31902 consensus_queue.cc:1035] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd8d1c9d9310417b9242bb3e16e4c556" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 42155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:42:26.482885 31895 consensus_queue.cc:1035] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f61d869461845e09b13896a0bbc4add" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 35789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:42:26.493516 31905 mvcc.cc:205] Tried to move back new op lower bound from 7316865623979208704 to 7316865623847432192. Current Snapshot: MvccSnapshot[applied={T|T < 7316865623979208704}]
I20260810 06:42:26.510249 31043 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:42:26.510510 31043 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:42:26.513996 31043 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b65071d0c60422d8267306639a1ffa0: Sending DeleteTablet for 3 replicas of tablet 94b3b93d6d08466ca645a6e4d9d9f476
I20260810 06:42:26.515147 31476 tablet_service.cc:1514] Processing DeleteTablet for tablet 94b3b93d6d08466ca645a6e4d9d9f476 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:42:26 UTC) from {username='slave'} at 127.0.0.1:37396
I20260810 06:42:26.515468 31249 tablet_service.cc:1514] Processing DeleteTablet for tablet 94b3b93d6d08466ca645a6e4d9d9f476 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:42:26 UTC) from {username='slave'} at 127.0.0.1:58734
I20260810 06:42:26.515617 31736 tablet_service.cc:1514] Processing DeleteTablet for tablet 94b3b93d6d08466ca645a6e4d9d9f476 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:42:26 UTC) from {username='slave'} at 127.0.0.1:58614
I20260810 06:42:26.517269 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 31146
I20260810 06:42:26.526052 31922 tablet_replica.cc:330] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add: stopping tablet replica
I20260810 06:42:26.526238 31919 tablet_replica.cc:330] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556: stopping tablet replica
I20260810 06:42:26.526683 31919 raft_consensus.cc:2229] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:42:26.526683 31922 raft_consensus.cc:2229] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:42:26.527192 31919 raft_consensus.cc:2258] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:42:26.527192 31922 raft_consensus.cc:2258] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:42:26.529351 31919 ts_tablet_manager.cc:1905] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:42:26.529827 31922 ts_tablet_manager.cc:1905] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:42:26.536337 31922 ts_tablet_manager.cc:1918] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:42:26.536337 31919 ts_tablet_manager.cc:1918] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:42:26.536556 31919 log.cc:1196] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/94b3b93d6d08466ca645a6e4d9d9f476
I20260810 06:42:26.536556 31922 log.cc:1196] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/94b3b93d6d08466ca645a6e4d9d9f476
I20260810 06:42:26.537071 31922 ts_tablet_manager.cc:1939] T 94b3b93d6d08466ca645a6e4d9d9f476 P 5f61d869461845e09b13896a0bbc4add: Deleting consensus metadata
I20260810 06:42:26.537177 31919 ts_tablet_manager.cc:1939] T 94b3b93d6d08466ca645a6e4d9d9f476 P cd8d1c9d9310417b9242bb3e16e4c556: Deleting consensus metadata
I20260810 06:42:26.538992 31022 catalog_manager.cc:4841] TS 5f61d869461845e09b13896a0bbc4add (127.31.59.195:35789): tablet 94b3b93d6d08466ca645a6e4d9d9f476 (table test-table [id=d13402c125f4480abf72b4bba14606b9]) successfully deleted
I20260810 06:42:26.539439 31022 catalog_manager.cc:4841] TS cd8d1c9d9310417b9242bb3e16e4c556 (127.31.59.194:42155): tablet 94b3b93d6d08466ca645a6e4d9d9f476 (table test-table [id=d13402c125f4480abf72b4bba14606b9]) successfully deleted
I20260810 06:42:26.540571 31920 tablet_replica.cc:330] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa: stopping tablet replica
I20260810 06:42:26.541051 31920 raft_consensus.cc:2229] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:42:26.541891 31920 raft_consensus.cc:2258] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:42:26.544167 31920 ts_tablet_manager.cc:1905] T 94b3b93d6d08466ca645a6e4d9d9f476 P 4cd6516fe1ee41dfb259c5621a6b7caa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:42:26.548673 31884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.518851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58762 (local address 127.31.59.193:33719)
0810 06:42:26.519048 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:26.519059 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:26.539895 (+ 20836us) server_negotiation.cc:407] Connection header received
0810 06:42:26.540451 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:26.540457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:26.540574 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:26.540681 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:26.541389 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.542645 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.544480 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.545126 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.545257 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:26.546702 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:26.546712 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:26.546729 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:26.547496 (+ 767us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:26.547555 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:26.547583 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:26.547694 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:26.548191 (+ 497us) server_negotiation.cc:299] Negotiation successful
0810 06:42:26.548287 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:42:26.549605 31365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:26.645306 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 31146
W20260810 06:42:26.675599 31022 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.59.193:33719 (error 108)
I20260810 06:42:26.676389 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 31381
I20260810 06:42:26.677780 31881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.677159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:33719 (local address 127.0.0.1:58766)
0810 06:42:26.677443 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:26.677598 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:33719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
W20260810 06:42:26.678066 31022 catalog_manager.cc:4584] TS 4cd6516fe1ee41dfb259c5621a6b7caa (127.31.59.193:33719): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 94b3b93d6d08466ca645a6e4d9d9f476: Network error: Client connection negotiation failed: client connection to 127.31.59.193:33719: connect: Connection refused (error 111)
I20260810 06:42:26.683058 31882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.677235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37402 (local address 127.31.59.194:42155)
0810 06:42:26.677443 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:26.677449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:26.677473 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:42:26.677625 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:26.677649 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:26.677769 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:26.677894 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:26.678368 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.679089 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.680319 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.680708 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.680791 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:26.681737 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:26.681746 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:26.681758 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:26.682376 (+ 618us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:26.682423 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:26.682443 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:26.682536 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:26.682727 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 06:42:26.682819 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 06:42:26.683694 31604 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:26.731878 31881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.731354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:33719 (local address 127.0.0.1:58778)
0810 06:42:26.731590 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:26.731709 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:33719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260810 06:42:26.732872 31881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.732540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:33719 (local address 127.0.0.1:58786)
0810 06:42:26.732656 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:26.732743 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:33719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 06:42:26.770277 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 31381
I20260810 06:42:26.794806 31881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.792817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:33719 (local address 127.0.0.1:58798)
0810 06:42:26.794457 (+ 1640us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:26.794584 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:33719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1438}
I20260810 06:42:26.795176 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 31618
I20260810 06:42:26.800858 31881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.800409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:33719 (local address 127.0.0.1:58814)
0810 06:42:26.800603 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:26.800706 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:33719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 06:42:26.806634 31886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.797042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58616 (local address 127.31.59.195:35789)
0810 06:42:26.799829 (+ 2787us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:26.799836 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:26.799857 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:42:26.799927 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:26.799933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:26.800041 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:26.800148 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:26.800705 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.801839 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.803193 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:26.803818 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:26.803939 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:26.805029 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:26.805049 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:26.805070 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:26.805851 (+ 781us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:26.805917 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:26.805943 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:26.806059 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:26.806160 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 06:42:26.806267 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2640}
I20260810 06:42:26.807331 31847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:26.881484 31881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.881051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:33719 (local address 127.0.0.1:58830)
0810 06:42:26.881243 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:26.881338 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:33719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 06:42:26.882473 31881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:26.882107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:33719 (local address 127.0.0.1:58842)
0810 06:42:26.882277 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:26.882371 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:33719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260810 06:42:26.896534 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 31618
I20260810 06:42:26.916651 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 30991
I20260810 06:42:26.917691 31094 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:27.026923 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 30991
2026-08-10T06:42:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2686 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:42:27 dist-test-slave-4tdj krb5kdc[31936](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:42:27 dist-test-slave-4tdj krb5kdc[31936](info): set up 2 sockets
Aug 10 06:42:27 dist-test-slave-4tdj krb5kdc[31936](info): commencing operation
krb5kdc: starting...
W20260810 06:42:28.271445 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:42:28 dist-test-slave-4tdj krb5kdc[31936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:42:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:42:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:42:28.541895 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:38305
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:37343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:38305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:28.868288 31957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:28.868566 31957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:28.868669 31957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:28.875638 31957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:42:28.875738 31957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:28.875792 31957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:28.875846 31957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:42:28.875890 31957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:42:28.885357 31957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:37343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:38305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:38305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:28.886628 31957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:28.887900 31957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:28.899118 31963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:28.899250 31965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:28.899273 31967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:28.900373 31957 server_base.cc:1034] running on GCE node
I20260810 06:42:28.901126 31957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:28.902554 31957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:28.903777 31957 hybrid_clock.cc:648] HybridClock initialized: now 1786344148903717 us; error 41 us; skew 500 ppm
I20260810 06:42:28.904135 31957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:28 dist-test-slave-4tdj krb5kdc[31936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344148, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:28.910331 31957 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:42:28.911887 31957 webserver.cc:460] Webserver started at http://127.31.59.254:45577/ using document root <none> and password file <none>
I20260810 06:42:28.912575 31957 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:28.912698 31957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:28.912990 31957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:28.916016 31957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "bae6a6e1f9e54ba187bc635f17b207f8"
format_stamp: "Formatted at 2026-08-10 06:42:28 on dist-test-slave-4tdj"
server_key: "fd2eedcdc2cfad58dafc7520b56ce434"
server_key_iv: "c5205927a09d6135d77d822ca315b4fa"
server_key_version: "encryptionkey@0"
I20260810 06:42:28.916793 31957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "bae6a6e1f9e54ba187bc635f17b207f8"
format_stamp: "Formatted at 2026-08-10 06:42:28 on dist-test-slave-4tdj"
server_key: "fd2eedcdc2cfad58dafc7520b56ce434"
server_key_iv: "c5205927a09d6135d77d822ca315b4fa"
server_key_version: "encryptionkey@0"
I20260810 06:42:28.921923 31957 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 06:42:28.925350 31989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:28.926688 31957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:42:28.926847 31957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "bae6a6e1f9e54ba187bc635f17b207f8"
format_stamp: "Formatted at 2026-08-10 06:42:28 on dist-test-slave-4tdj"
server_key: "fd2eedcdc2cfad58dafc7520b56ce434"
server_key_iv: "c5205927a09d6135d77d822ca315b4fa"
server_key_version: "encryptionkey@0"
I20260810 06:42:28.927053 31957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:28.941504 31957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:28.945472 31957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:28.945736 31957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:28.963685 31957 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:38305
I20260810 06:42:28.963707 32088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:38305 every 8 connection(s)
I20260810 06:42:28.964951 31957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:42:28.967746 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 31957
I20260810 06:42:28.967911 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:42:28.968339 31983 external_mini_cluster.cc:1428] Setting key d704c7e7e8e58772f0d65f0a9f46ce1e
I20260810 06:42:28.968603 32090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:28.982227 32090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8: Bootstrap starting.
Aug 10 06:42:28 dist-test-slave-4tdj krb5kdc[31936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:42:28.985756 32090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:28.987082 32090 log.cc:824] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:28.990397 32090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8: No bootstrap required, opened a new log
I20260810 06:42:28.992375 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:28.970068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54540 (local address 127.31.59.254:38305)
0810 06:42:28.970894 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:28.970908 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:28.970959 (+ 51us) server_negotiation.cc:407] Connection header received
0810 06:42:28.973284 (+ 2325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:28.973311 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:28.973783 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:28.974206 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:28.975597 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:28.977456 (+ 1859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:28.978820 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:28.979867 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:28.980079 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:28.983918 (+ 3839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:28.983929 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:28.983946 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:28.983999 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:28.987847 (+ 3848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:28.988696 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:28.988704 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:28.988711 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:28.988907 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:28.989281 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:28.989286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:28.989289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:28.989956 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:28.990276 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:28.990668 (+ 392us) server_negotiation.cc:299] Negotiation successful
0810 06:42:28.991105 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":347,"threads_started":1}
I20260810 06:42:28.998708 32090 raft_consensus.cc:348] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae6a6e1f9e54ba187bc635f17b207f8" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38305 } }
I20260810 06:42:28.999037 32090 raft_consensus.cc:374] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:28.999109 32090 raft_consensus.cc:729] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bae6a6e1f9e54ba187bc635f17b207f8, State: Initialized, Role: FOLLOWER
I20260810 06:42:28.999569 32090 consensus_queue.cc:260] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae6a6e1f9e54ba187bc635f17b207f8" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38305 } }
I20260810 06:42:28.999768 32090 raft_consensus.cc:388] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:42:28.999863 32090 raft_consensus.cc:482] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:42:29.000005 32090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:29.002250 32090 raft_consensus.cc:504] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae6a6e1f9e54ba187bc635f17b207f8" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38305 } }
I20260810 06:42:29.002744 32090 leader_election.cc:302] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bae6a6e1f9e54ba187bc635f17b207f8; no voters:
I20260810 06:42:29.003288 32090 leader_election.cc:288] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:42:29.003427 32099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:29.004092 32099 raft_consensus.cc:686] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8 [term 1 LEADER]: Becoming Leader. State: Replica: bae6a6e1f9e54ba187bc635f17b207f8, State: Running, Role: LEADER
I20260810 06:42:29.004632 32099 consensus_queue.cc:237] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae6a6e1f9e54ba187bc635f17b207f8" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38305 } }
I20260810 06:42:29.005200 32090 sys_catalog.cc:564] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:42:29.007599 32100 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bae6a6e1f9e54ba187bc635f17b207f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae6a6e1f9e54ba187bc635f17b207f8" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38305 } } }
I20260810 06:42:29.007575 32101 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bae6a6e1f9e54ba187bc635f17b207f8. Latest consensus state: current_term: 1 leader_uuid: "bae6a6e1f9e54ba187bc635f17b207f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae6a6e1f9e54ba187bc635f17b207f8" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38305 } } }
I20260810 06:42:29.008036 32100 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:29.008163 32101 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:29.009044 32110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:42:29.014896 32110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:42:29.024405 32110 catalog_manager.cc:1347] Generated new cluster ID: b0b7254997084de9a70a1fd07108093a
I20260810 06:42:29.024506 32110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:42:29.046710 32110 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:42:29.048585 32110 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:42:29.056160 32110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8: Generated new TSK 0
I20260810 06:42:29.056929 32110 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.59.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
Entry for principal kudu/127.31.59.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab.
I20260810 06:42:29.133538 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:38305
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:37343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:29.488175 32130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:29.488430 32130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:29.488574 32130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:29.496611 32130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:29.496727 32130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:29.496842 32130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:42:29.506547 32130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:37343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.193.keytab
--principal=kudu/127.31.59.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:38305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:29.507792 32130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:29.509069 32130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:29.521651 32136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:29.521624 32141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:29.521881 32138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:29.522436 32130 server_base.cc:1034] running on GCE node
I20260810 06:42:29.522994 32130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:29.523912 32130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:29.525105 32130 hybrid_clock.cc:648] HybridClock initialized: now 1786344149525052 us; error 50 us; skew 500 ppm
I20260810 06:42:29.525415 32130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:29 dist-test-slave-4tdj krb5kdc[31936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344149, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:29.528560 32130 init.cc:377] Logged in from keytab as kudu/127.31.59.193@KRBTEST.COM (short username kudu)
I20260810 06:42:29.530061 32130 webserver.cc:460] Webserver started at http://127.31.59.193:39145/ using document root <none> and password file <none>
I20260810 06:42:29.530779 32130 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:29.530900 32130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:29.531208 32130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:29.534088 32130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "c03f1df6f0a94a1eaac5c38c93098a89"
format_stamp: "Formatted at 2026-08-10 06:42:29 on dist-test-slave-4tdj"
server_key: "36d865929145e26fd97de3df0f8799df"
server_key_iv: "9c4c64e983d88559f2ed4f6f7a84584f"
server_key_version: "encryptionkey@0"
I20260810 06:42:29.534893 32130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "c03f1df6f0a94a1eaac5c38c93098a89"
format_stamp: "Formatted at 2026-08-10 06:42:29 on dist-test-slave-4tdj"
server_key: "36d865929145e26fd97de3df0f8799df"
server_key_iv: "9c4c64e983d88559f2ed4f6f7a84584f"
server_key_version: "encryptionkey@0"
I20260810 06:42:29.539813 32130 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:42:29.543067 32153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:29.544302 32130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:42:29.544453 32130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "c03f1df6f0a94a1eaac5c38c93098a89"
format_stamp: "Formatted at 2026-08-10 06:42:29 on dist-test-slave-4tdj"
server_key: "36d865929145e26fd97de3df0f8799df"
server_key_iv: "9c4c64e983d88559f2ed4f6f7a84584f"
server_key_version: "encryptionkey@0"
I20260810 06:42:29.544631 32130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:29.568096 32130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:29.572184 32130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:29.572449 32130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:29.573393 32130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:42:29.574743 32130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:42:29.574826 32130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:29.574940 32130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:42:29.575001 32130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:29.605736 32130 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:43841
I20260810 06:42:29.605803 32347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:43841 every 8 connection(s)
I20260810 06:42:29.607035 32130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:42:29.615284 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 32130
I20260810 06:42:29.615449 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:42:29.615880 31983 external_mini_cluster.cc:1428] Setting key 1cf24fb8bb6fc845f357c9f525adb3f5
Aug 10 06:42:29 dist-test-slave-4tdj krb5kdc[31936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344149, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.193@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:42:29.626591 32354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:29.609355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:59447 (local address 127.31.59.254:38305)
0810 06:42:29.609860 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:29.609864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:29.610784 (+ 920us) server_negotiation.cc:407] Connection header received
0810 06:42:29.612605 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:29.612611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:29.612691 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:29.612787 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:29.614226 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:29.615057 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:29.616654 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:29.617140 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:29.617258 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:29.620831 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:29.620837 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:29.620841 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:29.620877 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:29.623746 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:29.624409 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:29.624414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:29.624417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:29.624560 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:29.624926 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:29.624930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:29.624932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:29.625263 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:29.625400 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:29.626014 (+ 614us) server_negotiation.cc:299] Negotiation successful
0810 06:42:29.626232 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":293,"threads_started":1}
I20260810 06:42:29.627318 32356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:29.609890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:38305 (local address 127.31.59.193:59447)
0810 06:42:29.610636 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:29.610677 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:29.612331 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:29.613024 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:29.613032 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:29.613527 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:42:29.614053 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:29.614071 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:29.615223 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:29.615227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:29.616519 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:29.616532 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:29.617401 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:29.617410 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:29.617685 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:29.618398 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:42:29.618423 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:42:29.620613 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:42:29.623870 (+ 3257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:42:29.623876 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:42:29.623887 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:42:29.624299 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:42:29.624650 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:42:29.624653 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:42:29.624656 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:42:29.624833 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:42:29.625372 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:42:29.625377 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:42:29.625662 (+ 285us) client_negotiation.cc:769] Sending connection context
0810 06:42:29.625872 (+ 210us) client_negotiation.cc:240] Negotiation successful
0810 06:42:29.626228 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":256,"threads_started":1}
I20260810 06:42:29.628772 32350 heartbeater.cc:347] Connected to a master server at 127.31.59.254:38305
I20260810 06:42:29.629093 32350 heartbeater.cc:464] Registering TS with master...
I20260810 06:42:29.629921 32350 heartbeater.cc:511] Master 127.31.59.254:38305 requested a full tablet report, sending...
I20260810 06:42:29.632328 32035 ts_manager.cc:194] Registered new tserver with Master: c03f1df6f0a94a1eaac5c38c93098a89 (127.31.59.193:43841)
I20260810 06:42:29.634223 32035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.193@KRBTEST.COM'} at 127.31.59.193:59447
WARNING: no policy specified for kudu/127.31.59.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
Entry for principal kudu/127.31.59.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab.
I20260810 06:42:29.679577 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:38305
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:37343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:29.999856 32360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:30.000109 32360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:30.000211 32360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:30.007718 32360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:30.007841 32360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:30.007970 32360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:42:30.017741 32360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:37343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.194.keytab
--principal=kudu/127.31.59.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:38305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:30.018996 32360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:30.020303 32360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:30.032909 32371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:30.032948 32372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:30.033150 32375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:30.033864 32360 server_base.cc:1034] running on GCE node
I20260810 06:42:30.034479 32360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:30.035339 32360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:30.036548 32360 hybrid_clock.cc:648] HybridClock initialized: now 1786344150036515 us; error 55 us; skew 500 ppm
I20260810 06:42:30.036849 32360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:30 dist-test-slave-4tdj krb5kdc[31936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344150, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:30.039819 32360 init.cc:377] Logged in from keytab as kudu/127.31.59.194@KRBTEST.COM (short username kudu)
I20260810 06:42:30.041149 32360 webserver.cc:460] Webserver started at http://127.31.59.194:43867/ using document root <none> and password file <none>
I20260810 06:42:30.041783 32360 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:30.041894 32360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:30.042202 32360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:30.044971 32360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "c456a4e159634e518cf51d438f7d78cd"
format_stamp: "Formatted at 2026-08-10 06:42:30 on dist-test-slave-4tdj"
server_key: "8e3dace186a30f1245b0c1e8b22da64a"
server_key_iv: "ac1d588e3bd37755fa8c22c445b82339"
server_key_version: "encryptionkey@0"
I20260810 06:42:30.045728 32360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "c456a4e159634e518cf51d438f7d78cd"
format_stamp: "Formatted at 2026-08-10 06:42:30 on dist-test-slave-4tdj"
server_key: "8e3dace186a30f1245b0c1e8b22da64a"
server_key_iv: "ac1d588e3bd37755fa8c22c445b82339"
server_key_version: "encryptionkey@0"
I20260810 06:42:30.050865 32360 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260810 06:42:30.054164 32389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:30.055548 32360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:42:30.055703 32360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "c456a4e159634e518cf51d438f7d78cd"
format_stamp: "Formatted at 2026-08-10 06:42:30 on dist-test-slave-4tdj"
server_key: "8e3dace186a30f1245b0c1e8b22da64a"
server_key_iv: "ac1d588e3bd37755fa8c22c445b82339"
server_key_version: "encryptionkey@0"
I20260810 06:42:30.055878 32360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:30.071076 32360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:30.074798 32360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:30.075052 32360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:30.076056 32360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:42:30.077394 32360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:42:30.077474 32360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:30.077626 32360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:42:30.077685 32360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:30.108059 32360 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:35655
I20260810 06:42:30.108103 32607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:35655 every 8 connection(s)
I20260810 06:42:30.109331 32360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:42:30.115341 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 32360
I20260810 06:42:30.115547 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:42:30.116117 31983 external_mini_cluster.cc:1428] Setting key a41786cbac8925386f9aebc298078c60
Aug 10 06:42:30 dist-test-slave-4tdj krb5kdc[31936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344150, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.194@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:42:30.129825 32354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:30.111900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:55257 (local address 127.31.59.254:38305)
0810 06:42:30.112111 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:30.112117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:30.113353 (+ 1236us) server_negotiation.cc:407] Connection header received
0810 06:42:30.115182 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:30.115187 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:30.115260 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:30.115349 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:30.117216 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.117991 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.119634 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.120256 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.120386 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:30.123715 (+ 3329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:30.123721 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:30.123725 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:30.123762 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:30.126824 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:30.127534 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:30.127540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:30.127543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:30.127695 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:30.128185 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:30.128189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:30.128191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:30.128520 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:30.128650 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:30.129295 (+ 645us) server_negotiation.cc:299] Negotiation successful
0810 06:42:30.129509 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:42:30.130790 32611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:30.112462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:38305 (local address 127.31.59.194:55257)
0810 06:42:30.113176 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:30.113221 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:30.114889 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:30.115624 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:30.115635 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:30.116369 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:42:30.117046 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:30.117065 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.118162 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.118167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:30.119485 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:30.119498 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.120497 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.120502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:30.120705 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:30.121330 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:42:30.121353 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:42:30.123483 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:42:30.126955 (+ 3472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:42:30.126962 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:42:30.126975 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:42:30.127399 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:42:30.127815 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:42:30.127819 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:42:30.127822 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:42:30.128071 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:42:30.128627 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:42:30.128633 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:42:30.128947 (+ 314us) client_negotiation.cc:769] Sending connection context
0810 06:42:30.129168 (+ 221us) client_negotiation.cc:240] Negotiation successful
0810 06:42:30.129538 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":265,"threads_started":1}
I20260810 06:42:30.132373 32609 heartbeater.cc:347] Connected to a master server at 127.31.59.254:38305
I20260810 06:42:30.132681 32609 heartbeater.cc:464] Registering TS with master...
I20260810 06:42:30.133421 32609 heartbeater.cc:511] Master 127.31.59.254:38305 requested a full tablet report, sending...
I20260810 06:42:30.135032 32035 ts_manager.cc:194] Registered new tserver with Master: c456a4e159634e518cf51d438f7d78cd (127.31.59.194:35655)
I20260810 06:42:30.135841 32035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.194@KRBTEST.COM'} at 127.31.59.194:55257
WARNING: no policy specified for kudu/127.31.59.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
Entry for principal kudu/127.31.59.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab.
I20260810 06:42:30.182734 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:38305
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:37343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:30.507315 32617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:30.507572 32617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:30.507699 32617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:30.514740 32617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:30.514837 32617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:30.514981 32617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:42:30.524291 32617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:37343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.195.keytab
--principal=kudu/127.31.59.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:38305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:30.525552 32617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:30.526863 32617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:30.539525 32629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:30.539605 32632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:30.539642 32628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:30.540361 32617 server_base.cc:1034] running on GCE node
I20260810 06:42:30.540922 32617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:30.541785 32617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:30.542979 32617 hybrid_clock.cc:648] HybridClock initialized: now 1786344150542934 us; error 33 us; skew 500 ppm
I20260810 06:42:30.543275 32617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:30 dist-test-slave-4tdj krb5kdc[31936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344150, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:30.546221 32617 init.cc:377] Logged in from keytab as kudu/127.31.59.195@KRBTEST.COM (short username kudu)
I20260810 06:42:30.547585 32617 webserver.cc:460] Webserver started at http://127.31.59.195:41905/ using document root <none> and password file <none>
I20260810 06:42:30.548207 32617 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:30.548305 32617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:30.548552 32617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:30.551353 32617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "4375786d871f48868b0d7b7051c06b23"
format_stamp: "Formatted at 2026-08-10 06:42:30 on dist-test-slave-4tdj"
server_key: "cd1c387f59cd838cccd7ad6545807ddc"
server_key_iv: "2f14b4c9d8dc17f5505f0ae25c13620d"
server_key_version: "encryptionkey@0"
I20260810 06:42:30.552034 32617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "4375786d871f48868b0d7b7051c06b23"
format_stamp: "Formatted at 2026-08-10 06:42:30 on dist-test-slave-4tdj"
server_key: "cd1c387f59cd838cccd7ad6545807ddc"
server_key_iv: "2f14b4c9d8dc17f5505f0ae25c13620d"
server_key_version: "encryptionkey@0"
I20260810 06:42:30.556936 32617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:42:30.560216 32648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:30.561514 32617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:42:30.561653 32617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "4375786d871f48868b0d7b7051c06b23"
format_stamp: "Formatted at 2026-08-10 06:42:30 on dist-test-slave-4tdj"
server_key: "cd1c387f59cd838cccd7ad6545807ddc"
server_key_iv: "2f14b4c9d8dc17f5505f0ae25c13620d"
server_key_version: "encryptionkey@0"
I20260810 06:42:30.561832 32617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:30.587693 32617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:30.591645 32617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:30.591903 32617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:30.592788 32617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:42:30.594041 32617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:42:30.594127 32617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:30.594242 32617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:42:30.594314 32617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:30.625665 32617 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:34083
I20260810 06:42:30.625706 383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:34083 every 8 connection(s)
I20260810 06:42:30.626905 32617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:42:30.631665 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 32617
I20260810 06:42:30.631863 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:42:30.632426 31983 external_mini_cluster.cc:1428] Setting key e736125573e7a9a6e6fd874f6faa57f6
I20260810 06:42:30.637084 32350 heartbeater.cc:503] Master 127.31.59.254:38305 was elected leader, sending a full tablet report...
Aug 10 06:42:30 dist-test-slave-4tdj krb5kdc[31936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344150, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.195@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:42:30.646822 32354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:30.629350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:34243 (local address 127.31.59.254:38305)
0810 06:42:30.629540 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:30.629545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:30.630687 (+ 1142us) server_negotiation.cc:407] Connection header received
0810 06:42:30.633086 (+ 2399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:30.633092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:30.633207 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:30.633297 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:30.634954 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.635705 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.637349 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.637946 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.638074 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:30.641477 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:30.641485 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:30.641491 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:30.641535 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:30.644125 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:30.644770 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:30.644774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:30.644776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:30.644886 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:30.645232 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:30.645235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:30.645237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:30.645507 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:30.645646 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:30.646262 (+ 616us) server_negotiation.cc:299] Negotiation successful
0810 06:42:30.646492 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:42:30.647850 388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:30.629739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:38305 (local address 127.31.59.195:34243)
0810 06:42:30.630531 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:30.630575 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:30.632783 (+ 2208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:30.633545 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:30.633553 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:30.634190 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:42:30.634793 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:30.634811 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.635854 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.635858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:30.637213 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:30.637228 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.638200 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.638205 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:30.638461 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:30.639115 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:42:30.639138 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:42:30.641246 (+ 2108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:42:30.644248 (+ 3002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:42:30.644255 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:42:30.644265 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:42:30.644680 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:42:30.644957 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:42:30.644961 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:42:30.644963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:42:30.645141 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:42:30.645627 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:42:30.645632 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:42:30.645920 (+ 288us) client_negotiation.cc:769] Sending connection context
0810 06:42:30.646163 (+ 243us) client_negotiation.cc:240] Negotiation successful
0810 06:42:30.646658 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":287,"threads_started":1}
I20260810 06:42:30.649178 385 heartbeater.cc:347] Connected to a master server at 127.31.59.254:38305
I20260810 06:42:30.649497 385 heartbeater.cc:464] Registering TS with master...
I20260810 06:42:30.650188 385 heartbeater.cc:511] Master 127.31.59.254:38305 requested a full tablet report, sending...
I20260810 06:42:30.651568 32035 ts_manager.cc:194] Registered new tserver with Master: 4375786d871f48868b0d7b7051c06b23 (127.31.59.195:34083)
I20260810 06:42:30.652348 32035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.59.195@KRBTEST.COM'} at 127.31.59.195:34243
I20260810 06:42:30.659461 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:42:30.675204 32354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:30.663520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54542 (local address 127.31.59.254:38305)
0810 06:42:30.663676 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:30.663681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:30.664127 (+ 446us) server_negotiation.cc:407] Connection header received
0810 06:42:30.664254 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:30.664257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:30.664318 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:30.664394 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:30.665844 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.666716 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.667883 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.668267 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.668355 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:30.670515 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:30.670521 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:30.670524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:30.670554 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:30.673023 (+ 2469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:30.673607 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:30.673611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:30.673613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:30.673735 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:30.673997 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:30.674000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:30.674002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:30.674323 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:30.674509 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:30.674825 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 06:42:30.674979 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:42:30.678721 32035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:42:30.685323 32035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:42:30.708259 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:30.696597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:35655 (local address 127.0.0.1:49008)
0810 06:42:30.698239 (+ 1642us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:30.698280 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:30.698472 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:30.700040 (+ 1568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:30.700048 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:30.700095 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:30.700484 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:30.700496 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.702770 (+ 2274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.702777 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:30.705173 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:30.705190 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.707300 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.707306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:30.707527 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:30.707584 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 06:42:30.707691 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 06:42:30.707890 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1370,"thread_start_us":2167,"threads_started":1}
I20260810 06:42:30.710089 405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:30.696876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49008 (local address 127.31.59.194:35655)
0810 06:42:30.699383 (+ 2507us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:30.699391 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:30.699419 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:42:30.699527 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:30.699536 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:30.699788 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:30.700708 (+ 920us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:30.700940 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.702606 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.705416 (+ 2810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.707027 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.709264 (+ 2237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:30.709485 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:30.709567 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 06:42:30.709739 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2343,"thread_start_us":2398,"threads_started":1}
I20260810 06:42:30.714363 32489 tablet_service.cc:1467] Processing CreateTablet for tablet f5bb01a6a1bb4f5ea17507645c026840 (DEFAULT_TABLE table=test-table [id=c4e2c055d71143e1a0c6737415c65ef1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:42:30.715751 32489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5bb01a6a1bb4f5ea17507645c026840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:30.717278 404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:30.696947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:43841 (local address 127.0.0.1:54270)
0810 06:42:30.699064 (+ 2117us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:30.699080 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:30.699179 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:30.710967 (+ 11788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:30.710972 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:30.710997 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:30.711411 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:30.711421 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.712840 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.712845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:30.714734 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:30.714747 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.716475 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.716482 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:30.716642 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:30.716678 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:42:30.716736 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 06:42:30.716865 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1974,"mutex_wait_us":103,"thread_start_us":1441,"threads_started":1}
I20260810 06:42:30.717411 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:30.699178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:34083 (local address 127.0.0.1:59300)
0810 06:42:30.708961 (+ 9783us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:30.708980 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:30.709094 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:30.710869 (+ 1775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:30.710874 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:30.710898 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:30.711090 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:30.711101 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.713029 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.713034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:30.714978 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:30.714992 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.716732 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.716737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:30.716895 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:30.716923 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 06:42:30.716985 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 06:42:30.717107 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9686,"mutex_wait_us":27,"thread_start_us":11970,"threads_started":1}
I20260810 06:42:30.718408 418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:30.699535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59300 (local address 127.31.59.195:34083)
0810 06:42:30.710022 (+ 10487us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:30.710030 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:30.710064 (+ 34us) server_negotiation.cc:407] Connection header received
0810 06:42:30.710167 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:30.710175 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:30.710434 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:30.710884 (+ 450us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:30.711302 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.712844 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.715162 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.716566 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.717604 (+ 1038us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:30.717831 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:30.717909 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 06:42:30.718058 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10328,"thread_start_us":10165,"threads_started":1}
I20260810 06:42:30.718873 411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:30.697611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54270 (local address 127.31.59.193:43841)
0810 06:42:30.710419 (+ 12808us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:30.710427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:30.710454 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:42:30.710548 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:30.710554 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:30.710760 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:30.711586 (+ 826us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:30.711747 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.712690 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.714881 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.716251 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.717784 (+ 1533us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:30.718339 (+ 555us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:30.718418 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 06:42:30.718576 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12601,"thread_start_us":12983,"threads_started":1}
I20260810 06:42:30.722405 32242 tablet_service.cc:1467] Processing CreateTablet for tablet f5bb01a6a1bb4f5ea17507645c026840 (DEFAULT_TABLE table=test-table [id=c4e2c055d71143e1a0c6737415c65ef1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:42:30.722877 32736 tablet_service.cc:1467] Processing CreateTablet for tablet f5bb01a6a1bb4f5ea17507645c026840 (DEFAULT_TABLE table=test-table [id=c4e2c055d71143e1a0c6737415c65ef1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:42:30.723966 32242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5bb01a6a1bb4f5ea17507645c026840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:30.724306 32736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5bb01a6a1bb4f5ea17507645c026840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:30.732390 420 tablet_bootstrap.cc:492] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd: Bootstrap starting.
I20260810 06:42:30.736155 422 tablet_bootstrap.cc:492] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23: Bootstrap starting.
I20260810 06:42:30.736862 420 tablet_bootstrap.cc:654] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:30.737653 424 tablet_bootstrap.cc:492] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89: Bootstrap starting.
I20260810 06:42:30.738158 420 log.cc:824] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:30.739497 422 tablet_bootstrap.cc:654] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:30.740643 424 tablet_bootstrap.cc:654] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:30.740695 422 log.cc:824] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:30.741462 420 tablet_bootstrap.cc:492] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd: No bootstrap required, opened a new log
I20260810 06:42:30.741658 424 log.cc:824] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:30.741773 420 ts_tablet_manager.cc:1397] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 06:42:30.744271 422 tablet_bootstrap.cc:492] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23: No bootstrap required, opened a new log
I20260810 06:42:30.744500 422 ts_tablet_manager.cc:1397] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 06:42:30.744510 424 tablet_bootstrap.cc:492] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89: No bootstrap required, opened a new log
I20260810 06:42:30.744760 424 ts_tablet_manager.cc:1397] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 06:42:30.751886 422 raft_consensus.cc:348] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c456a4e159634e518cf51d438f7d78cd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35655 } } peers { permanent_uuid: "c03f1df6f0a94a1eaac5c38c93098a89" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 43841 } } peers { permanent_uuid: "4375786d871f48868b0d7b7051c06b23" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 34083 } }
I20260810 06:42:30.751847 420 raft_consensus.cc:348] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c456a4e159634e518cf51d438f7d78cd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35655 } } peers { permanent_uuid: "c03f1df6f0a94a1eaac5c38c93098a89" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 43841 } } peers { permanent_uuid: "4375786d871f48868b0d7b7051c06b23" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 34083 } }
I20260810 06:42:30.752236 422 raft_consensus.cc:374] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:30.752244 420 raft_consensus.cc:374] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:30.752331 420 raft_consensus.cc:729] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c456a4e159634e518cf51d438f7d78cd, State: Initialized, Role: FOLLOWER
I20260810 06:42:30.752388 422 raft_consensus.cc:729] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4375786d871f48868b0d7b7051c06b23, State: Initialized, Role: FOLLOWER
I20260810 06:42:30.753005 422 consensus_queue.cc:260] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c456a4e159634e518cf51d438f7d78cd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35655 } } peers { permanent_uuid: "c03f1df6f0a94a1eaac5c38c93098a89" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 43841 } } peers { permanent_uuid: "4375786d871f48868b0d7b7051c06b23" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 34083 } }
I20260810 06:42:30.752939 420 consensus_queue.cc:260] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c456a4e159634e518cf51d438f7d78cd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35655 } } peers { permanent_uuid: "c03f1df6f0a94a1eaac5c38c93098a89" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 43841 } } peers { permanent_uuid: "4375786d871f48868b0d7b7051c06b23" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 34083 } }
I20260810 06:42:30.754001 385 heartbeater.cc:503] Master 127.31.59.254:38305 was elected leader, sending a full tablet report...
I20260810 06:42:30.754520 32609 heartbeater.cc:503] Master 127.31.59.254:38305 was elected leader, sending a full tablet report...
I20260810 06:42:30.754789 424 raft_consensus.cc:348] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c456a4e159634e518cf51d438f7d78cd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35655 } } peers { permanent_uuid: "c03f1df6f0a94a1eaac5c38c93098a89" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 43841 } } peers { permanent_uuid: "4375786d871f48868b0d7b7051c06b23" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 34083 } }
I20260810 06:42:30.755191 420 ts_tablet_manager.cc:1428] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260810 06:42:30.755200 424 raft_consensus.cc:374] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:30.755290 424 raft_consensus.cc:729] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c03f1df6f0a94a1eaac5c38c93098a89, State: Initialized, Role: FOLLOWER
I20260810 06:42:30.755947 424 consensus_queue.cc:260] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c456a4e159634e518cf51d438f7d78cd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35655 } } peers { permanent_uuid: "c03f1df6f0a94a1eaac5c38c93098a89" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 43841 } } peers { permanent_uuid: "4375786d871f48868b0d7b7051c06b23" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 34083 } }
I20260810 06:42:30.756332 422 ts_tablet_manager.cc:1428] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260810 06:42:30.757701 424 ts_tablet_manager.cc:1428] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260810 06:42:30.822477 433 raft_consensus.cc:482] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:42:30.822803 433 raft_consensus.cc:504] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c456a4e159634e518cf51d438f7d78cd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35655 } } peers { permanent_uuid: "c03f1df6f0a94a1eaac5c38c93098a89" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 43841 } } peers { permanent_uuid: "4375786d871f48868b0d7b7051c06b23" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 34083 } }
I20260810 06:42:30.824433 433 leader_election.cc:288] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c03f1df6f0a94a1eaac5c38c93098a89 (127.31.59.193:43841), 4375786d871f48868b0d7b7051c06b23 (127.31.59.195:34083)
I20260810 06:42:30.831444 411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:30.824836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:45041 (local address 127.31.59.193:43841)
0810 06:42:30.825065 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:30.825072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:30.825559 (+ 487us) server_negotiation.cc:407] Connection header received
0810 06:42:30.825778 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:30.825784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:30.825866 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:30.825958 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:30.826592 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.827496 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.829458 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.830507 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.830645 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:30.830697 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:30.831073 (+ 376us) server_negotiation.cc:299] Negotiation successful
0810 06:42:30.831198 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:42:30.831629 437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:30.824805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:43841 (local address 127.31.59.194:45041)
0810 06:42:30.825440 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:30.825456 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:30.825626 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:30.826023 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:30.826028 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:30.826060 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:30.826439 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:30.826452 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.827626 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.827632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:30.829302 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:30.829315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.830712 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.830719 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:30.830879 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:30.830913 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 06:42:30.831036 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 06:42:30.831236 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":20,"thread_start_us":343,"threads_started":1}
I20260810 06:42:30.832018 438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:30.825587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:34083 (local address 127.31.59.194:51359)
0810 06:42:30.826154 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:30.826168 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:30.826270 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:30.826616 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:30.826621 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:30.826645 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:30.827001 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:30.827012 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.828139 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.828142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:30.829803 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:30.829818 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.831211 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.831218 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:30.831402 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:30.831439 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 06:42:30.831548 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 06:42:30.831677 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":19,"thread_start_us":305,"threads_started":1}
I20260810 06:42:30.832170 418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:30.825742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:51359 (local address 127.31.59.195:34083)
0810 06:42:30.825899 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:30.825903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:30.826252 (+ 349us) server_negotiation.cc:407] Connection header received
0810 06:42:30.826406 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:30.826411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:30.826486 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:30.826579 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:30.827131 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.828005 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.829957 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.830988 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.831131 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:30.831196 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:30.831821 (+ 625us) server_negotiation.cc:299] Negotiation successful
0810 06:42:30.831941 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 06:42:30.832718 32272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5bb01a6a1bb4f5ea17507645c026840" candidate_uuid: "c456a4e159634e518cf51d438f7d78cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c03f1df6f0a94a1eaac5c38c93098a89" is_pre_election: true
I20260810 06:42:30.833099 306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5bb01a6a1bb4f5ea17507645c026840" candidate_uuid: "c456a4e159634e518cf51d438f7d78cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4375786d871f48868b0d7b7051c06b23" is_pre_election: true
I20260810 06:42:30.833330 32272 raft_consensus.cc:2454] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c456a4e159634e518cf51d438f7d78cd in term 0.
I20260810 06:42:30.833636 306 raft_consensus.cc:2454] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c456a4e159634e518cf51d438f7d78cd in term 0.
I20260810 06:42:30.834062 32390 leader_election.cc:302] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c03f1df6f0a94a1eaac5c38c93098a89, c456a4e159634e518cf51d438f7d78cd; no voters:
I20260810 06:42:30.834438 433 raft_consensus.cc:2781] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:42:30.834596 433 raft_consensus.cc:482] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:42:30.834707 433 raft_consensus.cc:3037] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:30.836884 433 raft_consensus.cc:504] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c456a4e159634e518cf51d438f7d78cd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35655 } } peers { permanent_uuid: "c03f1df6f0a94a1eaac5c38c93098a89" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 43841 } } peers { permanent_uuid: "4375786d871f48868b0d7b7051c06b23" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 34083 } }
I20260810 06:42:30.837430 433 leader_election.cc:288] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [CANDIDATE]: Term 1 election: Requested vote from peers c03f1df6f0a94a1eaac5c38c93098a89 (127.31.59.193:43841), 4375786d871f48868b0d7b7051c06b23 (127.31.59.195:34083)
I20260810 06:42:30.837877 32272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5bb01a6a1bb4f5ea17507645c026840" candidate_uuid: "c456a4e159634e518cf51d438f7d78cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c03f1df6f0a94a1eaac5c38c93098a89"
I20260810 06:42:30.837926 306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5bb01a6a1bb4f5ea17507645c026840" candidate_uuid: "c456a4e159634e518cf51d438f7d78cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4375786d871f48868b0d7b7051c06b23"
I20260810 06:42:30.838056 32272 raft_consensus.cc:3037] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:30.838119 306 raft_consensus.cc:3037] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:30.840250 32272 raft_consensus.cc:2454] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c456a4e159634e518cf51d438f7d78cd in term 1.
I20260810 06:42:30.840343 306 raft_consensus.cc:2454] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c456a4e159634e518cf51d438f7d78cd in term 1.
I20260810 06:42:30.840605 32390 leader_election.cc:302] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c03f1df6f0a94a1eaac5c38c93098a89, c456a4e159634e518cf51d438f7d78cd; no voters:
I20260810 06:42:30.840883 433 raft_consensus.cc:2781] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:30.841306 433 raft_consensus.cc:686] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [term 1 LEADER]: Becoming Leader. State: Replica: c456a4e159634e518cf51d438f7d78cd, State: Running, Role: LEADER
I20260810 06:42:30.841810 433 consensus_queue.cc:237] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c456a4e159634e518cf51d438f7d78cd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35655 } } peers { permanent_uuid: "c03f1df6f0a94a1eaac5c38c93098a89" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 43841 } } peers { permanent_uuid: "4375786d871f48868b0d7b7051c06b23" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 34083 } }
I20260810 06:42:30.846215 32031 catalog_manager.cc:5495] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd reported cstate change: term changed from 0 to 1, leader changed from <none> to c456a4e159634e518cf51d438f7d78cd (127.31.59.194). New cstate: current_term: 1 leader_uuid: "c456a4e159634e518cf51d438f7d78cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c456a4e159634e518cf51d438f7d78cd" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 35655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c03f1df6f0a94a1eaac5c38c93098a89" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 43841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4375786d871f48868b0d7b7051c06b23" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 34083 } health_report { overall_health: UNKNOWN } } }
W20260810 06:42:30.859313 32352 tablet.cc:2369] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:42:30.861227 32610 tablet.cc:2369] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:42:30.878437 386 tablet.cc:2369] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:42:30.896934 405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:30.892000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49012 (local address 127.31.59.194:35655)
0810 06:42:30.892201 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:30.892206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:30.892291 (+ 85us) server_negotiation.cc:407] Connection header received
0810 06:42:30.892463 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:30.892467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:30.892549 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:30.892677 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:42:30.892956 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.893688 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.895082 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.895483 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.895580 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:30.895959 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:42:30.896442 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:42:30.896559 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:30.896657 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 06:42:30.896724 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:42:30.905055 306 raft_consensus.cc:1261] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23 [term 1 FOLLOWER]: Refusing update from remote peer c456a4e159634e518cf51d438f7d78cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:42:30.905109 32272 raft_consensus.cc:1261] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89 [term 1 FOLLOWER]: Refusing update from remote peer c456a4e159634e518cf51d438f7d78cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:42:30.906015 439 consensus_queue.cc:1035] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4375786d871f48868b0d7b7051c06b23" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 34083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:42:30.906405 433 consensus_queue.cc:1035] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c03f1df6f0a94a1eaac5c38c93098a89" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 43841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:42:30.916270 445 mvcc.cc:205] Tried to move back new op lower bound from 7316865642097168384 to 7316865641852149760. Current Snapshot: MvccSnapshot[applied={T|T < 7316865642097168384}]
I20260810 06:42:30.934732 32032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:42:30.935016 32032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:42:30.938589 32032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bae6a6e1f9e54ba187bc635f17b207f8: Sending DeleteTablet for 3 replicas of tablet f5bb01a6a1bb4f5ea17507645c026840
I20260810 06:42:30.939719 32489 tablet_service.cc:1514] Processing DeleteTablet for tablet f5bb01a6a1bb4f5ea17507645c026840 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:42:30 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:49008
I20260810 06:42:30.940138 32242 tablet_service.cc:1514] Processing DeleteTablet for tablet f5bb01a6a1bb4f5ea17507645c026840 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:42:30 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:54270
I20260810 06:42:30.940892 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 32130
I20260810 06:42:30.940863 32736 tablet_service.cc:1514] Processing DeleteTablet for tablet f5bb01a6a1bb4f5ea17507645c026840 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:42:30 UTC) from {username='kudu', principal='kudu/127.31.59.254@KRBTEST.COM'} at 127.0.0.1:59300
I20260810 06:42:30.955067 459 tablet_replica.cc:330] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89: stopping tablet replica
I20260810 06:42:30.955596 459 raft_consensus.cc:2229] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:42:30.956173 459 raft_consensus.cc:2258] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:42:30.956995 462 tablet_replica.cc:330] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23: stopping tablet replica
I20260810 06:42:30.957257 458 tablet_replica.cc:330] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd: stopping tablet replica
I20260810 06:42:30.957432 462 raft_consensus.cc:2229] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:42:30.957654 458 raft_consensus.cc:2229] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:42:30.957913 462 raft_consensus.cc:2258] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:42:30.958345 458 raft_consensus.cc:2258] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:42:30.960616 458 ts_tablet_manager.cc:1905] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:42:30.961243 459 ts_tablet_manager.cc:1905] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:42:30.968974 458 ts_tablet_manager.cc:1918] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:42:30.969154 458 log.cc:1196] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/f5bb01a6a1bb4f5ea17507645c026840
I20260810 06:42:30.969713 458 ts_tablet_manager.cc:1939] T f5bb01a6a1bb4f5ea17507645c026840 P c456a4e159634e518cf51d438f7d78cd: Deleting consensus metadata
I20260810 06:42:30.970916 462 ts_tablet_manager.cc:1905] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:42:30.971566 31996 catalog_manager.cc:4841] TS c456a4e159634e518cf51d438f7d78cd (127.31.59.194:35655): tablet f5bb01a6a1bb4f5ea17507645c026840 (table test-table [id=c4e2c055d71143e1a0c6737415c65ef1]) successfully deleted
I20260810 06:42:30.972685 459 ts_tablet_manager.cc:1918] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:42:30.972852 459 log.cc:1196] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-0/wal/wals/f5bb01a6a1bb4f5ea17507645c026840
I20260810 06:42:30.973374 459 ts_tablet_manager.cc:1939] T f5bb01a6a1bb4f5ea17507645c026840 P c03f1df6f0a94a1eaac5c38c93098a89: Deleting consensus metadata
Aug 10 06:42:30 dist-test-slave-4tdj krb5kdc[31936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.193@KRBTEST.COM
I20260810 06:42:30.975198 31993 catalog_manager.cc:4841] TS c03f1df6f0a94a1eaac5c38c93098a89 (127.31.59.193:43841): tablet f5bb01a6a1bb4f5ea17507645c026840 (table test-table [id=c4e2c055d71143e1a0c6737415c65ef1]) successfully deleted
I20260810 06:42:30.977286 462 ts_tablet_manager.cc:1918] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:42:30.977454 462 log.cc:1196] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/f5bb01a6a1bb4f5ea17507645c026840
I20260810 06:42:30.977980 462 ts_tablet_manager.cc:1939] T f5bb01a6a1bb4f5ea17507645c026840 P 4375786d871f48868b0d7b7051c06b23: Deleting consensus metadata
I20260810 06:42:30.979769 31993 catalog_manager.cc:4841] TS 4375786d871f48868b0d7b7051c06b23 (127.31.59.195:34083): tablet f5bb01a6a1bb4f5ea17507645c026840 (table test-table [id=c4e2c055d71143e1a0c6737415c65ef1]) successfully deleted
I20260810 06:42:30.983402 411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:30.942348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54272 (local address 127.31.59.193:43841)
0810 06:42:30.942534 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:30.942541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:30.952579 (+ 10038us) server_negotiation.cc:407] Connection header received
0810 06:42:30.953054 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:30.953061 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:30.953172 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:30.953245 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:30.959462 (+ 6217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.960691 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.968087 (+ 7396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:30.968782 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:30.968916 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:30.975550 (+ 6634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:30.975567 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:30.975584 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:30.975630 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:30.979436 (+ 3806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:30.980331 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:30.980341 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:30.980345 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:30.980516 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:30.980935 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:30.980940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:30.980943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:30.981363 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:30.981643 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:30.982552 (+ 909us) server_negotiation.cc:299] Negotiation successful
0810 06:42:30.982995 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:42:30.984092 32338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:31.077135 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 32130
I20260810 06:42:31.109875 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 32360
Aug 10 06:42:31 dist-test-slave-4tdj krb5kdc[31936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.194@KRBTEST.COM
I20260810 06:42:31.131054 405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:31.110774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49022 (local address 127.31.59.194:35655)
0810 06:42:31.111406 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:31.111412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:31.111430 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:42:31.111486 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:31.111492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:31.111587 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:31.111693 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:31.113581 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:31.114964 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:31.116887 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:31.117576 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:31.117693 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:31.123399 (+ 5706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:31.123415 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:31.123436 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:31.123498 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:31.127801 (+ 4303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:31.128647 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:31.128656 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:31.128660 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:31.128857 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:31.129300 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:31.129305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:31.129309 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:31.129793 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:31.130120 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:31.130500 (+ 380us) server_negotiation.cc:299] Negotiation successful
0810 06:42:31.130717 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492}
I20260810 06:42:31.131850 32603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:31.244263 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 32360
I20260810 06:42:31.266788 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 32617
Aug 10 06:42:31 dist-test-slave-4tdj krb5kdc[31936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.195@KRBTEST.COM
I20260810 06:42:31.281474 418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:31.267501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59314 (local address 127.31.59.195:34083)
0810 06:42:31.267701 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:31.267707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:31.267728 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:42:31.267914 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:31.267919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:31.267985 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:31.268095 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:31.269394 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:31.270099 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:31.271402 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:31.271799 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:31.271912 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:31.276402 (+ 4490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:31.276419 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:31.276436 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:31.276472 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:31.279222 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:31.279843 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:31.279849 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:31.279852 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:31.279982 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:31.280273 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:31.280275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:31.280277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:31.280600 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:31.280820 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:31.281060 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 06:42:31.281225 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 06:42:31.282100 376 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:31.368721 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 32617
I20260810 06:42:31.393220 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 31957
I20260810 06:42:31.394317 32079 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:31.503019 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 31957
2026-08-10T06:42:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4476 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T06:42:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:42:31Z Disabled control of system clock
I20260810 06:42:31.590745 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:39971
--webserver_interface=127.31.59.254
--webserver_port=0
--builtin_ntp_servers=127.31.59.212:44221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:39971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:42:31.914314 480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:31.914584 480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:31.914707 480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:31.921670 480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:42:31.921767 480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:31.921826 480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:31.921901 480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:42:31.921957 480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:42:31.930932 480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:44221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:39971
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:39971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:31.932202 480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:31.933979 480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:31.944787 494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:31.944942 492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:31.944936 488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:31.945806 480 server_base.cc:1034] running on GCE node
I20260810 06:42:31.946558 480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:31.947835 480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:31.949031 480 hybrid_clock.cc:648] HybridClock initialized: now 1786344151948981 us; error 39 us; skew 500 ppm
I20260810 06:42:31.949373 480 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:42:31.951508 480 webserver.cc:460] Webserver started at http://127.31.59.254:42393/ using document root <none> and password file <none>
I20260810 06:42:31.952176 480 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:31.952267 480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:31.952560 480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:31.955375 480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "a26100fcf4c449239d3f3826f3f54879"
format_stamp: "Formatted at 2026-08-10 06:42:31 on dist-test-slave-4tdj"
server_key: "e653bc33ede5b6021f39069c1e51dc64"
server_key_iv: "588b71741a1b5f9cdb0fbfe649ae7254"
server_key_version: "encryptionkey@0"
I20260810 06:42:31.956144 480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "a26100fcf4c449239d3f3826f3f54879"
format_stamp: "Formatted at 2026-08-10 06:42:31 on dist-test-slave-4tdj"
server_key: "e653bc33ede5b6021f39069c1e51dc64"
server_key_iv: "588b71741a1b5f9cdb0fbfe649ae7254"
server_key_version: "encryptionkey@0"
I20260810 06:42:31.960985 480 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:42:31.964411 506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:31.965770 480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:42:31.965929 480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "a26100fcf4c449239d3f3826f3f54879"
format_stamp: "Formatted at 2026-08-10 06:42:31 on dist-test-slave-4tdj"
server_key: "e653bc33ede5b6021f39069c1e51dc64"
server_key_iv: "588b71741a1b5f9cdb0fbfe649ae7254"
server_key_version: "encryptionkey@0"
I20260810 06:42:31.966140 480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:31.979169 480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:31.979858 480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:31.980105 480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:31.997119 480 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:39971
I20260810 06:42:31.997149 586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:39971 every 8 connection(s)
I20260810 06:42:31.998363 480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:42:32.001835 587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:32.003494 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 480
I20260810 06:42:32.003656 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:42:32.004050 31983 external_mini_cluster.cc:1428] Setting key cc799619c7cf9c2835132cb6347bf64e
I20260810 06:42:32.013298 587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879: Bootstrap starting.
I20260810 06:42:32.016716 587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:32.017257 593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:32.005815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44702 (local address 127.31.59.254:39971)
0810 06:42:32.006917 (+ 1102us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:32.006927 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:32.006970 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:42:32.008633 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:32.008654 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:32.009063 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:32.009350 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:32.009952 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:32.011316 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:32.013095 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:32.013790 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:32.013910 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:32.015112 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:32.015119 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:32.015131 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:32.015637 (+ 506us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:32.015723 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:32.015744 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:32.015828 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:32.016048 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 06:42:32.016321 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":414,"threads_started":1}
I20260810 06:42:32.017774 587 log.cc:824] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:32.020713 587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879: No bootstrap required, opened a new log
I20260810 06:42:32.027851 587 raft_consensus.cc:348] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26100fcf4c449239d3f3826f3f54879" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 39971 } }
I20260810 06:42:32.028187 587 raft_consensus.cc:374] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:32.028275 587 raft_consensus.cc:729] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a26100fcf4c449239d3f3826f3f54879, State: Initialized, Role: FOLLOWER
I20260810 06:42:32.028781 587 consensus_queue.cc:260] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26100fcf4c449239d3f3826f3f54879" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 39971 } }
I20260810 06:42:32.028995 587 raft_consensus.cc:388] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:42:32.029167 587 raft_consensus.cc:482] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:42:32.029301 587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:32.031752 587 raft_consensus.cc:504] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26100fcf4c449239d3f3826f3f54879" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 39971 } }
I20260810 06:42:32.032241 587 leader_election.cc:302] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a26100fcf4c449239d3f3826f3f54879; no voters:
I20260810 06:42:32.032738 587 leader_election.cc:288] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:42:32.032881 595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:32.033517 595 raft_consensus.cc:686] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879 [term 1 LEADER]: Becoming Leader. State: Replica: a26100fcf4c449239d3f3826f3f54879, State: Running, Role: LEADER
I20260810 06:42:32.034041 595 consensus_queue.cc:237] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26100fcf4c449239d3f3826f3f54879" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 39971 } }
I20260810 06:42:32.034646 587 sys_catalog.cc:564] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:42:32.036545 596 sys_catalog.cc:455] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a26100fcf4c449239d3f3826f3f54879" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26100fcf4c449239d3f3826f3f54879" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 39971 } } }
I20260810 06:42:32.036967 596 sys_catalog.cc:458] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:32.037344 597 sys_catalog.cc:455] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a26100fcf4c449239d3f3826f3f54879. Latest consensus state: current_term: 1 leader_uuid: "a26100fcf4c449239d3f3826f3f54879" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26100fcf4c449239d3f3826f3f54879" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 39971 } } }
I20260810 06:42:32.037618 597 sys_catalog.cc:458] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:32.039884 608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:42:32.043989 608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:42:32.051892 608 catalog_manager.cc:1347] Generated new cluster ID: 0edbd01e2366416f9e307e9e93553f65
I20260810 06:42:32.052001 608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:42:32.089025 608 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:42:32.090899 608 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:42:32.114198 608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879: Generated new TSK 0
I20260810 06:42:32.115260 608 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:42:32.130584 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.193:0
--local_ip_for_outbound_sockets=127.31.59.193
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:39971
--builtin_ntp_servers=127.31.59.212:44221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:42:32.461964 627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:32.462240 627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:32.462385 627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:32.469409 627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:32.469511 627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:32.469660 627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.193
I20260810 06:42:32.478366 627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:44221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.59.193
--webserver_port=0
--tserver_master_addrs=127.31.59.254:39971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.193
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:32.479635 627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:32.481346 627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:32.493875 633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:32.493813 637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:32.493996 641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:32.494603 627 server_base.cc:1034] running on GCE node
I20260810 06:42:32.495167 627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:32.496037 627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:32.497248 627 hybrid_clock.cc:648] HybridClock initialized: now 1786344152497198 us; error 30 us; skew 500 ppm
I20260810 06:42:32.497548 627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:42:32.499635 627 webserver.cc:460] Webserver started at http://127.31.59.193:43889/ using document root <none> and password file <none>
I20260810 06:42:32.500267 627 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:32.500370 627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:32.500669 627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:32.503427 627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-0/data/instance:
uuid: "b144b2b9d65b4415a271af890cd12357"
format_stamp: "Formatted at 2026-08-10 06:42:32 on dist-test-slave-4tdj"
server_key: "b037eca0c9dc2b2488ab48f7392f9e7e"
server_key_iv: "0bf49041dfa315a1328de4529a0eabb7"
server_key_version: "encryptionkey@0"
I20260810 06:42:32.504169 627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance:
uuid: "b144b2b9d65b4415a271af890cd12357"
format_stamp: "Formatted at 2026-08-10 06:42:32 on dist-test-slave-4tdj"
server_key: "b037eca0c9dc2b2488ab48f7392f9e7e"
server_key_iv: "0bf49041dfa315a1328de4529a0eabb7"
server_key_version: "encryptionkey@0"
I20260810 06:42:32.509014 627 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 06:42:32.512307 652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:32.513505 627 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:42:32.513661 627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-0/wal
uuid: "b144b2b9d65b4415a271af890cd12357"
format_stamp: "Formatted at 2026-08-10 06:42:32 on dist-test-slave-4tdj"
server_key: "b037eca0c9dc2b2488ab48f7392f9e7e"
server_key_iv: "0bf49041dfa315a1328de4529a0eabb7"
server_key_version: "encryptionkey@0"
I20260810 06:42:32.513854 627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:32.535449 627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:32.536216 627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:32.536500 627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:32.537401 627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:42:32.538865 627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:42:32.538981 627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:32.539088 627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:42:32.539165 627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:32.567907 627 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.193:45133
I20260810 06:42:32.567976 833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.193:45133 every 8 connection(s)
I20260810 06:42:32.569178 627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-0/data/info.pb
I20260810 06:42:32.570919 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 627
I20260810 06:42:32.571182 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-0/wal/instance
I20260810 06:42:32.571741 31983 external_mini_cluster.cc:1428] Setting key 9a1dc68ae3f6010ea28162dd1305b454
I20260810 06:42:32.574847 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.194:0
--local_ip_for_outbound_sockets=127.31.59.194
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:39971
--builtin_ntp_servers=127.31.59.212:44221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:42:32.582504 836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:32.572059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.193:45683 (local address 127.31.59.254:39971)
0810 06:42:32.572546 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:32.572552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:32.573843 (+ 1291us) server_negotiation.cc:407] Connection header received
0810 06:42:32.575855 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:32.575860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:32.575938 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:32.576049 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:32.577107 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:32.578089 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:32.579719 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:32.580242 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:32.580355 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:32.581666 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:32.581672 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:32.581676 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:32.581696 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:32.581745 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:32.581759 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:32.581841 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:32.582021 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 06:42:32.582112 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":243,"threads_started":1}
I20260810 06:42:32.583405 837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:32.572606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:39971 (local address 127.31.59.193:45683)
0810 06:42:32.573672 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:32.573731 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:32.575577 (+ 1846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:32.576247 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:32.576255 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:32.576338 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:42:32.576939 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:32.576956 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:32.578234 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:32.578238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:32.579574 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:32.579587 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:32.580438 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:32.580444 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:32.580682 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:32.581348 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:42:32.581351 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:42:32.581390 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:42:32.581475 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:42:32.581850 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:42:32.581856 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:42:32.581862 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 06:42:32.581994 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 06:42:32.582273 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":297,"threads_started":1}
I20260810 06:42:32.584842 834 heartbeater.cc:347] Connected to a master server at 127.31.59.254:39971
I20260810 06:42:32.585132 834 heartbeater.cc:464] Registering TS with master...
I20260810 06:42:32.585855 834 heartbeater.cc:511] Master 127.31.59.254:39971 requested a full tablet report, sending...
I20260810 06:42:32.587862 536 ts_manager.cc:194] Registered new tserver with Master: b144b2b9d65b4415a271af890cd12357 (127.31.59.193:45133)
I20260810 06:42:32.589308 536 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.193:45683
W20260810 06:42:32.911280 838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:32.911556 838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:32.911679 838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:32.918901 838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:32.919003 838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:32.919148 838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.194
I20260810 06:42:32.928366 838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:44221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.59.194
--webserver_port=0
--tserver_master_addrs=127.31.59.254:39971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.194
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:32.929684 838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:32.931489 838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:32.944751 846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:32.944726 847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:32.944792 849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:32.945771 838 server_base.cc:1034] running on GCE node
I20260810 06:42:32.946450 838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:32.947439 838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:32.948659 838 hybrid_clock.cc:648] HybridClock initialized: now 1786344152948599 us; error 40 us; skew 500 ppm
I20260810 06:42:32.949036 838 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:42:32.951498 838 webserver.cc:460] Webserver started at http://127.31.59.194:38859/ using document root <none> and password file <none>
I20260810 06:42:32.952210 838 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:32.952333 838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:32.952677 838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:32.955662 838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-1/data/instance:
uuid: "74eac40dcb43433f816f89470c28b145"
format_stamp: "Formatted at 2026-08-10 06:42:32 on dist-test-slave-4tdj"
server_key: "e166acc3c2a455dc08b7acab08421cb7"
server_key_iv: "a0399c491cb6f23577d947cfcc5d39f8"
server_key_version: "encryptionkey@0"
I20260810 06:42:32.956485 838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance:
uuid: "74eac40dcb43433f816f89470c28b145"
format_stamp: "Formatted at 2026-08-10 06:42:32 on dist-test-slave-4tdj"
server_key: "e166acc3c2a455dc08b7acab08421cb7"
server_key_iv: "a0399c491cb6f23577d947cfcc5d39f8"
server_key_version: "encryptionkey@0"
I20260810 06:42:32.961676 838 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260810 06:42:32.965176 857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:32.966485 838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:42:32.966655 838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-1/wal
uuid: "74eac40dcb43433f816f89470c28b145"
format_stamp: "Formatted at 2026-08-10 06:42:32 on dist-test-slave-4tdj"
server_key: "e166acc3c2a455dc08b7acab08421cb7"
server_key_iv: "a0399c491cb6f23577d947cfcc5d39f8"
server_key_version: "encryptionkey@0"
I20260810 06:42:32.966856 838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:32.983119 838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:32.983917 838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:32.984234 838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:32.985205 838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:42:32.986769 838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:42:32.986877 838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:32.986996 838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:42:32.987078 838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:33.018059 838 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.194:36279
I20260810 06:42:33.018121 1055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.194:36279 every 8 connection(s)
I20260810 06:42:33.019351 838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-1/data/info.pb
I20260810 06:42:33.028672 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 838
I20260810 06:42:33.028862 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-1/wal/instance
I20260810 06:42:33.029389 31983 external_mini_cluster.cc:1428] Setting key cb4c86e9e88e7ff6229d86812268369d
I20260810 06:42:33.032176 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.59.195:0
--local_ip_for_outbound_sockets=127.31.59.195
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:39971
--builtin_ntp_servers=127.31.59.212:44221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:42:33.034586 836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:33.022731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:41895 (local address 127.31.59.254:39971)
0810 06:42:33.022977 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:33.022983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:33.024727 (+ 1744us) server_negotiation.cc:407] Connection header received
0810 06:42:33.027049 (+ 2322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:33.027054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:33.027125 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:33.027211 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:33.028728 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.029802 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.031701 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.032116 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.032246 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:33.033616 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:33.033622 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:33.033626 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:33.033645 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:33.033679 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:33.033692 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:33.033774 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:33.034007 (+ 233us) server_negotiation.cc:299] Negotiation successful
0810 06:42:33.034115 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:42:33.035607 1062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:33.023269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:39971 (local address 127.31.59.194:41895)
0810 06:42:33.024551 (+ 1282us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:33.024601 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:33.026766 (+ 2165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:33.027513 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:33.027527 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:33.027636 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:42:33.028487 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:33.028515 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.029981 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.029986 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:33.031558 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:33.031572 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.032358 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.032366 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:33.032595 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:33.033289 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:42:33.033292 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:42:33.033332 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:42:33.033412 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:42:33.033786 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:42:33.033798 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:42:33.033808 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 06:42:33.034001 (+ 193us) client_negotiation.cc:240] Negotiation successful
0810 06:42:33.034429 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":497,"threads_started":1}
I20260810 06:42:33.037061 1056 heartbeater.cc:347] Connected to a master server at 127.31.59.254:39971
I20260810 06:42:33.037361 1056 heartbeater.cc:464] Registering TS with master...
I20260810 06:42:33.038141 1056 heartbeater.cc:511] Master 127.31.59.254:39971 requested a full tablet report, sending...
I20260810 06:42:33.039712 536 ts_manager.cc:194] Registered new tserver with Master: 74eac40dcb43433f816f89470c28b145 (127.31.59.194:36279)
I20260810 06:42:33.040681 536 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.194:41895
W20260810 06:42:33.362187 1063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:33.362478 1063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:33.362613 1063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:33.369800 1063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:33.369886 1063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:33.370033 1063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.59.195
I20260810 06:42:33.379020 1063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:44221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.59.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.59.195
--webserver_port=0
--tserver_master_addrs=127.31.59.254:39971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.59.195
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:33.380250 1063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:33.381927 1063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:33.394531 1073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:33.394537 1077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:33.394440 1074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:33.395221 1063 server_base.cc:1034] running on GCE node
I20260810 06:42:33.395789 1063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:33.396610 1063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:33.397806 1063 hybrid_clock.cc:648] HybridClock initialized: now 1786344153397755 us; error 45 us; skew 500 ppm
I20260810 06:42:33.398087 1063 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:42:33.400189 1063 webserver.cc:460] Webserver started at http://127.31.59.195:37281/ using document root <none> and password file <none>
I20260810 06:42:33.400801 1063 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:33.400898 1063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:33.401185 1063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:33.403883 1063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-2/data/instance:
uuid: "e18f7d5bd11e4c139786e4f70e97e0a4"
format_stamp: "Formatted at 2026-08-10 06:42:33 on dist-test-slave-4tdj"
server_key: "ca6ad7f50edf50e0068dc3297afa8fbc"
server_key_iv: "07486401a75f6ff3863ddc80e5675297"
server_key_version: "encryptionkey@0"
I20260810 06:42:33.404754 1063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance:
uuid: "e18f7d5bd11e4c139786e4f70e97e0a4"
format_stamp: "Formatted at 2026-08-10 06:42:33 on dist-test-slave-4tdj"
server_key: "ca6ad7f50edf50e0068dc3297afa8fbc"
server_key_iv: "07486401a75f6ff3863ddc80e5675297"
server_key_version: "encryptionkey@0"
I20260810 06:42:33.410095 1063 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 06:42:33.413466 1085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:33.414727 1063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:42:33.414858 1063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-2/wal
uuid: "e18f7d5bd11e4c139786e4f70e97e0a4"
format_stamp: "Formatted at 2026-08-10 06:42:33 on dist-test-slave-4tdj"
server_key: "ca6ad7f50edf50e0068dc3297afa8fbc"
server_key_iv: "07486401a75f6ff3863ddc80e5675297"
server_key_version: "encryptionkey@0"
I20260810 06:42:33.415072 1063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:33.427589 1063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:33.428519 1063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:33.428856 1063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:33.430019 1063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:42:33.431921 1063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:42:33.432029 1063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:33.432152 1063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:42:33.432227 1063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:33.464021 1063 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.195:37321
I20260810 06:42:33.464072 1282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.195:37321 every 8 connection(s)
I20260810 06:42:33.465344 1063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-2/data/info.pb
I20260810 06:42:33.471119 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 1063
I20260810 06:42:33.471277 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-2/wal/instance
I20260810 06:42:33.471896 31983 external_mini_cluster.cc:1428] Setting key e040fddf24f57aca2ca7e90350d0a596
I20260810 06:42:33.477695 836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:33.467754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.195:42695 (local address 127.31.59.254:39971)
0810 06:42:33.467924 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:33.467929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:33.469301 (+ 1372us) server_negotiation.cc:407] Connection header received
0810 06:42:33.471117 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:33.471122 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:33.471210 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:33.471304 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:33.472636 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.473482 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.475426 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.475826 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.475915 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:33.477103 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:33.477108 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:33.477111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:33.477127 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:33.477156 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:33.477167 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:33.477233 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:33.477388 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 06:42:33.477460 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:42:33.478614 1288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:33.468264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:39971 (local address 127.31.59.195:42695)
0810 06:42:33.469157 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:33.469192 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:33.470855 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:33.471603 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:33.471616 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:33.471731 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:42:33.472469 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:33.472491 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.473650 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.473654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:33.475291 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:33.475303 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.475995 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.475999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:33.476192 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:33.476825 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:42:33.476828 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:42:33.476866 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:42:33.476935 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:42:33.477237 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:42:33.477243 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:42:33.477248 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:42:33.477373 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 06:42:33.477634 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":266,"threads_started":1}
I20260810 06:42:33.479732 1283 heartbeater.cc:347] Connected to a master server at 127.31.59.254:39971
I20260810 06:42:33.479969 1283 heartbeater.cc:464] Registering TS with master...
I20260810 06:42:33.480614 1283 heartbeater.cc:511] Master 127.31.59.254:39971 requested a full tablet report, sending...
I20260810 06:42:33.481808 536 ts_manager.cc:194] Registered new tserver with Master: e18f7d5bd11e4c139786e4f70e97e0a4 (127.31.59.195:37321)
I20260810 06:42:33.482638 536 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.59.195:42695
I20260810 06:42:33.490712 31983 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:42:33.499568 836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:33.494471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44704 (local address 127.31.59.254:39971)
0810 06:42:33.494648 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:33.494654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:33.495106 (+ 452us) server_negotiation.cc:407] Connection header received
0810 06:42:33.495239 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:33.495243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:33.495299 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:33.495366 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:33.495781 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.496455 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.497659 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.498064 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.498148 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:33.499013 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:33.499018 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:33.499021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:33.499036 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:33.499076 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:33.499088 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:33.499153 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:33.499233 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 06:42:33.499300 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:42:33.502852 536 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:42:33.509577 536 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:42:33.532317 1321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:33.521281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.194:36279 (local address 127.0.0.1:39230)
0810 06:42:33.523107 (+ 1826us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:33.523145 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:33.523323 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:33.524710 (+ 1387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:33.524719 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:33.524758 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:33.525254 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:33.525271 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.527280 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.527286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:33.529437 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:33.529448 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.531393 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.531399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:33.531570 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:33.531625 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 06:42:33.531750 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 06:42:33.531930 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1621,"mutex_wait_us":123,"thread_start_us":2303,"threads_started":1}
I20260810 06:42:33.532547 1316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:33.521469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:37321 (local address 127.0.0.1:51970)
0810 06:42:33.523841 (+ 2372us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:33.523858 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:33.523985 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:33.524892 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:33.524895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:33.524920 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:33.525323 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:33.525343 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.527326 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.527334 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:33.529938 (+ 2604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:33.529952 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.531738 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.531744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:33.531919 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:33.531952 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:42:33.532032 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:42:33.532153 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2253,"mutex_wait_us":183,"spinlock_wait_cycles":4480,"thread_start_us":1321,"threads_started":1}
I20260810 06:42:33.532619 1322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:33.522091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51970 (local address 127.31.59.195:37321)
0810 06:42:33.522709 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:33.522718 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:33.524097 (+ 1379us) server_negotiation.cc:407] Connection header received
0810 06:42:33.524206 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:33.524213 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:33.524422 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:33.524618 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:33.525523 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.527139 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.530140 (+ 3001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.531525 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.531670 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:33.531918 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:33.532104 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 06:42:33.532298 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":294,"threads_started":1}
I20260810 06:42:33.533392 1319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:33.521450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39230 (local address 127.31.59.194:36279)
0810 06:42:33.521988 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:33.522005 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:33.523263 (+ 1258us) server_negotiation.cc:407] Connection header received
0810 06:42:33.523556 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:33.523565 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:33.523816 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:33.524008 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:33.525480 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.527154 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.529619 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.530847 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.530974 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:33.531272 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:33.532851 (+ 1579us) server_negotiation.cc:299] Negotiation successful
0810 06:42:33.533016 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":257,"threads_started":1}
I20260810 06:42:33.534696 1324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:33.523720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45133 (local address 127.0.0.1:54076)
0810 06:42:33.525662 (+ 1942us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:33.525675 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:33.525780 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:33.526242 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:33.526246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:33.526271 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:33.526670 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:33.526682 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.528446 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.528452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:33.530456 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:33.530470 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.533764 (+ 3294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.533771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:33.533943 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:33.533975 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:42:33.534064 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 06:42:33.534183 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1851,"mutex_wait_us":2,"thread_start_us":1719,"threads_started":1}
I20260810 06:42:33.534734 1326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:33.524183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54076 (local address 127.31.59.193:45133)
0810 06:42:33.524730 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:33.524738 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:33.525765 (+ 1027us) server_negotiation.cc:407] Connection header received
0810 06:42:33.525915 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:33.525920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:33.526083 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:33.526769 (+ 686us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:33.526975 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.528283 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.530678 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.531939 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.532084 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:33.532311 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:33.534139 (+ 1828us) server_negotiation.cc:299] Negotiation successful
0810 06:42:33.534390 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":269,"threads_started":1}
I20260810 06:42:33.536728 1170 tablet_service.cc:1467] Processing CreateTablet for tablet 75ab638b1a5d43e98971b5ac35b225ef (DEFAULT_TABLE table=test-table [id=db8a1508c5de470691999074e8df8eef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:42:33.536968 948 tablet_service.cc:1467] Processing CreateTablet for tablet 75ab638b1a5d43e98971b5ac35b225ef (DEFAULT_TABLE table=test-table [id=db8a1508c5de470691999074e8df8eef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:42:33.537966 1170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75ab638b1a5d43e98971b5ac35b225ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:33.538321 948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75ab638b1a5d43e98971b5ac35b225ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:33.538277 731 tablet_service.cc:1467] Processing CreateTablet for tablet 75ab638b1a5d43e98971b5ac35b225ef (DEFAULT_TABLE table=test-table [id=db8a1508c5de470691999074e8df8eef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:42:33.539613 731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75ab638b1a5d43e98971b5ac35b225ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:33.549015 1332 tablet_bootstrap.cc:492] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4: Bootstrap starting.
I20260810 06:42:33.552211 1332 tablet_bootstrap.cc:654] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:33.552328 1333 tablet_bootstrap.cc:492] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145: Bootstrap starting.
I20260810 06:42:33.554255 1334 tablet_bootstrap.cc:492] T 75ab638b1a5d43e98971b5ac35b225ef P b144b2b9d65b4415a271af890cd12357: Bootstrap starting.
I20260810 06:42:33.554605 1332 log.cc:824] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:33.555663 1333 tablet_bootstrap.cc:654] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:33.556748 1333 log.cc:824] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:33.557286 1334 tablet_bootstrap.cc:654] T 75ab638b1a5d43e98971b5ac35b225ef P b144b2b9d65b4415a271af890cd12357: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:33.557796 1332 tablet_bootstrap.cc:492] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4: No bootstrap required, opened a new log
I20260810 06:42:33.558025 1332 ts_tablet_manager.cc:1397] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260810 06:42:33.558444 1334 log.cc:824] T 75ab638b1a5d43e98971b5ac35b225ef P b144b2b9d65b4415a271af890cd12357: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:33.559787 1333 tablet_bootstrap.cc:492] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145: No bootstrap required, opened a new log
I20260810 06:42:33.560041 1333 ts_tablet_manager.cc:1397] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260810 06:42:33.560909 1334 tablet_bootstrap.cc:492] T 75ab638b1a5d43e98971b5ac35b225ef P b144b2b9d65b4415a271af890cd12357: No bootstrap required, opened a new log
I20260810 06:42:33.561190 1334 ts_tablet_manager.cc:1397] T 75ab638b1a5d43e98971b5ac35b225ef P b144b2b9d65b4415a271af890cd12357: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 06:42:33.568070 1334 raft_consensus.cc:348] T 75ab638b1a5d43e98971b5ac35b225ef P b144b2b9d65b4415a271af890cd12357 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74eac40dcb43433f816f89470c28b145" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36279 } } peers { permanent_uuid: "e18f7d5bd11e4c139786e4f70e97e0a4" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37321 } } peers { permanent_uuid: "b144b2b9d65b4415a271af890cd12357" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45133 } }
I20260810 06:42:33.568087 1332 raft_consensus.cc:348] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74eac40dcb43433f816f89470c28b145" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36279 } } peers { permanent_uuid: "e18f7d5bd11e4c139786e4f70e97e0a4" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37321 } } peers { permanent_uuid: "b144b2b9d65b4415a271af890cd12357" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45133 } }
I20260810 06:42:33.568482 1334 raft_consensus.cc:374] T 75ab638b1a5d43e98971b5ac35b225ef P b144b2b9d65b4415a271af890cd12357 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:33.568493 1332 raft_consensus.cc:374] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:33.568572 1334 raft_consensus.cc:729] T 75ab638b1a5d43e98971b5ac35b225ef P b144b2b9d65b4415a271af890cd12357 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b144b2b9d65b4415a271af890cd12357, State: Initialized, Role: FOLLOWER
I20260810 06:42:33.568579 1332 raft_consensus.cc:729] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e18f7d5bd11e4c139786e4f70e97e0a4, State: Initialized, Role: FOLLOWER
I20260810 06:42:33.569275 1334 consensus_queue.cc:260] T 75ab638b1a5d43e98971b5ac35b225ef P b144b2b9d65b4415a271af890cd12357 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74eac40dcb43433f816f89470c28b145" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36279 } } peers { permanent_uuid: "e18f7d5bd11e4c139786e4f70e97e0a4" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37321 } } peers { permanent_uuid: "b144b2b9d65b4415a271af890cd12357" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45133 } }
I20260810 06:42:33.569669 1332 consensus_queue.cc:260] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74eac40dcb43433f816f89470c28b145" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36279 } } peers { permanent_uuid: "e18f7d5bd11e4c139786e4f70e97e0a4" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37321 } } peers { permanent_uuid: "b144b2b9d65b4415a271af890cd12357" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45133 } }
I20260810 06:42:33.570060 1333 raft_consensus.cc:348] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74eac40dcb43433f816f89470c28b145" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36279 } } peers { permanent_uuid: "e18f7d5bd11e4c139786e4f70e97e0a4" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37321 } } peers { permanent_uuid: "b144b2b9d65b4415a271af890cd12357" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45133 } }
I20260810 06:42:33.570509 1333 raft_consensus.cc:374] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:33.570636 1333 raft_consensus.cc:729] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74eac40dcb43433f816f89470c28b145, State: Initialized, Role: FOLLOWER
I20260810 06:42:33.571327 1333 consensus_queue.cc:260] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74eac40dcb43433f816f89470c28b145" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36279 } } peers { permanent_uuid: "e18f7d5bd11e4c139786e4f70e97e0a4" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37321 } } peers { permanent_uuid: "b144b2b9d65b4415a271af890cd12357" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45133 } }
I20260810 06:42:33.574787 1332 ts_tablet_manager.cc:1428] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4: Time spent starting tablet: real 0.017s user 0.016s sys 0.001s
I20260810 06:42:33.575162 1283 heartbeater.cc:503] Master 127.31.59.254:39971 was elected leader, sending a full tablet report...
I20260810 06:42:33.579238 1333 ts_tablet_manager.cc:1428] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145: Time spent starting tablet: real 0.019s user 0.016s sys 0.003s
I20260810 06:42:33.579581 1056 heartbeater.cc:503] Master 127.31.59.254:39971 was elected leader, sending a full tablet report...
I20260810 06:42:33.581746 834 heartbeater.cc:503] Master 127.31.59.254:39971 was elected leader, sending a full tablet report...
I20260810 06:42:33.582451 1334 ts_tablet_manager.cc:1428] T 75ab638b1a5d43e98971b5ac35b225ef P b144b2b9d65b4415a271af890cd12357: Time spent starting tablet: real 0.021s user 0.011s sys 0.012s
W20260810 06:42:33.716998 1287 tablet.cc:2369] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:42:33.773046 1061 tablet.cc:2369] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:42:33.821574 835 tablet.cc:2369] T 75ab638b1a5d43e98971b5ac35b225ef P b144b2b9d65b4415a271af890cd12357: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:42:33.841046 1343 raft_consensus.cc:482] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:42:33.841413 1343 raft_consensus.cc:504] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74eac40dcb43433f816f89470c28b145" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36279 } } peers { permanent_uuid: "e18f7d5bd11e4c139786e4f70e97e0a4" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37321 } } peers { permanent_uuid: "b144b2b9d65b4415a271af890cd12357" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45133 } }
I20260810 06:42:33.843173 1343 leader_election.cc:288] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e18f7d5bd11e4c139786e4f70e97e0a4 (127.31.59.195:37321), b144b2b9d65b4415a271af890cd12357 (127.31.59.193:45133)
I20260810 06:42:33.849700 1322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:33.843659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:56519 (local address 127.31.59.195:37321)
0810 06:42:33.843868 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:33.843873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:33.844273 (+ 400us) server_negotiation.cc:407] Connection header received
0810 06:42:33.844510 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:33.844517 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:33.844627 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:33.844725 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:33.845347 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.846109 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.847998 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.848763 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.848909 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:33.848980 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:33.849314 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 06:42:33.849441 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:42:33.849826 1344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:33.843569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.195:37321 (local address 127.31.59.194:56519)
0810 06:42:33.844159 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:33.844176 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:33.844325 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:33.844772 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:33.844776 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:33.844803 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:33.845192 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:33.845201 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.846235 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.846240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:33.847813 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:33.847825 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.848939 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.848945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:33.849112 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:33.849180 (+ 68us) client_negotiation.cc:769] Sending connection context
0810 06:42:33.849260 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:42:33.849442 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":149,"spinlock_wait_cycles":33152,"thread_start_us":315,"threads_started":1}
I20260810 06:42:33.849836 1326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:33.844492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.59.194:57191 (local address 127.31.59.193:45133)
0810 06:42:33.844709 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:33.844715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:33.844934 (+ 219us) server_negotiation.cc:407] Connection header received
0810 06:42:33.845115 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:33.845119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:33.845188 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:33.845274 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:42:33.845781 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.846495 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.848170 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.848929 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.849062 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:33.849128 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:33.849464 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 06:42:33.849599 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 06:42:33.849936 1346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:33.844270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45133 (local address 127.31.59.194:57191)
0810 06:42:33.844832 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:33.844849 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:42:33.844984 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:42:33.845313 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:42:33.845317 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:42:33.845340 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:42:33.845637 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:33.845645 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.846621 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.846626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:33.847988 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:42:33.848000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.849117 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.849123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:42:33.849278 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:33.849340 (+ 62us) client_negotiation.cc:769] Sending connection context
0810 06:42:33.849421 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:42:33.849537 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":5,"thread_start_us":287,"threads_started":1}
I20260810 06:42:33.850879 1209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75ab638b1a5d43e98971b5ac35b225ef" candidate_uuid: "74eac40dcb43433f816f89470c28b145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e18f7d5bd11e4c139786e4f70e97e0a4" is_pre_election: true
I20260810 06:42:33.850914 763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75ab638b1a5d43e98971b5ac35b225ef" candidate_uuid: "74eac40dcb43433f816f89470c28b145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b144b2b9d65b4415a271af890cd12357" is_pre_election: true
I20260810 06:42:33.851363 1209 raft_consensus.cc:2454] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74eac40dcb43433f816f89470c28b145 in term 0.
I20260810 06:42:33.851363 763 raft_consensus.cc:2454] T 75ab638b1a5d43e98971b5ac35b225ef P b144b2b9d65b4415a271af890cd12357 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74eac40dcb43433f816f89470c28b145 in term 0.
I20260810 06:42:33.851939 870 leader_election.cc:302] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74eac40dcb43433f816f89470c28b145, e18f7d5bd11e4c139786e4f70e97e0a4; no voters:
I20260810 06:42:33.852276 1343 raft_consensus.cc:2781] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:42:33.852407 1343 raft_consensus.cc:482] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:42:33.852514 1343 raft_consensus.cc:3037] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:33.854705 1343 raft_consensus.cc:504] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74eac40dcb43433f816f89470c28b145" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36279 } } peers { permanent_uuid: "e18f7d5bd11e4c139786e4f70e97e0a4" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37321 } } peers { permanent_uuid: "b144b2b9d65b4415a271af890cd12357" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45133 } }
I20260810 06:42:33.855239 1343 leader_election.cc:288] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [CANDIDATE]: Term 1 election: Requested vote from peers e18f7d5bd11e4c139786e4f70e97e0a4 (127.31.59.195:37321), b144b2b9d65b4415a271af890cd12357 (127.31.59.193:45133)
I20260810 06:42:33.855645 1209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75ab638b1a5d43e98971b5ac35b225ef" candidate_uuid: "74eac40dcb43433f816f89470c28b145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e18f7d5bd11e4c139786e4f70e97e0a4"
I20260810 06:42:33.855746 763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75ab638b1a5d43e98971b5ac35b225ef" candidate_uuid: "74eac40dcb43433f816f89470c28b145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b144b2b9d65b4415a271af890cd12357"
I20260810 06:42:33.855813 1209 raft_consensus.cc:3037] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:33.855937 763 raft_consensus.cc:3037] T 75ab638b1a5d43e98971b5ac35b225ef P b144b2b9d65b4415a271af890cd12357 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:33.857905 1209 raft_consensus.cc:2454] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74eac40dcb43433f816f89470c28b145 in term 1.
I20260810 06:42:33.858070 763 raft_consensus.cc:2454] T 75ab638b1a5d43e98971b5ac35b225ef P b144b2b9d65b4415a271af890cd12357 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74eac40dcb43433f816f89470c28b145 in term 1.
I20260810 06:42:33.858285 870 leader_election.cc:302] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74eac40dcb43433f816f89470c28b145, e18f7d5bd11e4c139786e4f70e97e0a4; no voters:
I20260810 06:42:33.858593 1343 raft_consensus.cc:2781] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:33.858996 1343 raft_consensus.cc:686] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [term 1 LEADER]: Becoming Leader. State: Replica: 74eac40dcb43433f816f89470c28b145, State: Running, Role: LEADER
I20260810 06:42:33.859449 1343 consensus_queue.cc:237] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74eac40dcb43433f816f89470c28b145" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36279 } } peers { permanent_uuid: "e18f7d5bd11e4c139786e4f70e97e0a4" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37321 } } peers { permanent_uuid: "b144b2b9d65b4415a271af890cd12357" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45133 } }
I20260810 06:42:33.864100 533 catalog_manager.cc:5495] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74eac40dcb43433f816f89470c28b145 (127.31.59.194). New cstate: current_term: 1 leader_uuid: "74eac40dcb43433f816f89470c28b145" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74eac40dcb43433f816f89470c28b145" member_type: VOTER last_known_addr { host: "127.31.59.194" port: 36279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e18f7d5bd11e4c139786e4f70e97e0a4" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b144b2b9d65b4415a271af890cd12357" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45133 } health_report { overall_health: UNKNOWN } } }
I20260810 06:42:33.907343 1319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:33.902280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39236 (local address 127.31.59.194:36279)
0810 06:42:33.902527 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:33.902532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:33.902553 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:42:33.902754 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:33.902758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:33.902838 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:33.902969 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:42:33.903267 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.903964 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.905338 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.905741 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.905835 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:33.906268 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:42:33.906824 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:42:33.906936 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:33.907071 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 06:42:33.907148 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:42:33.915712 763 raft_consensus.cc:1261] T 75ab638b1a5d43e98971b5ac35b225ef P b144b2b9d65b4415a271af890cd12357 [term 1 FOLLOWER]: Refusing update from remote peer 74eac40dcb43433f816f89470c28b145: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:42:33.915756 1209 raft_consensus.cc:1261] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4 [term 1 FOLLOWER]: Refusing update from remote peer 74eac40dcb43433f816f89470c28b145: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:42:33.916700 1349 consensus_queue.cc:1035] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e18f7d5bd11e4c139786e4f70e97e0a4" member_type: VOTER last_known_addr { host: "127.31.59.195" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:42:33.917054 1343 consensus_queue.cc:1035] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b144b2b9d65b4415a271af890cd12357" member_type: VOTER last_known_addr { host: "127.31.59.193" port: 45133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:42:33.932628 1358 mvcc.cc:205] Tried to move back new op lower bound from 7316865654428680192 to 7316865654211833856. Current Snapshot: MvccSnapshot[applied={T|T < 7316865654428680192}]
I20260810 06:42:33.941234 536 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:42:33.941501 536 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:42:33.945034 536 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a26100fcf4c449239d3f3826f3f54879: Sending DeleteTablet for 3 replicas of tablet 75ab638b1a5d43e98971b5ac35b225ef
I20260810 06:42:33.946184 948 tablet_service.cc:1514] Processing DeleteTablet for tablet 75ab638b1a5d43e98971b5ac35b225ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:42:33 UTC) from {username='slave'} at 127.0.0.1:39230
I20260810 06:42:33.946234 731 tablet_service.cc:1514] Processing DeleteTablet for tablet 75ab638b1a5d43e98971b5ac35b225ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:42:33 UTC) from {username='slave'} at 127.0.0.1:54076
I20260810 06:42:33.947141 1170 tablet_service.cc:1514] Processing DeleteTablet for tablet 75ab638b1a5d43e98971b5ac35b225ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:42:33 UTC) from {username='slave'} at 127.0.0.1:51970
I20260810 06:42:33.947278 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 627
I20260810 06:42:33.956115 1375 tablet_replica.cc:330] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4: stopping tablet replica
I20260810 06:42:33.957224 1375 raft_consensus.cc:2229] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:42:33.958467 1375 raft_consensus.cc:2258] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:42:33.961637 1375 ts_tablet_manager.cc:1905] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:42:33.964148 1373 tablet_replica.cc:330] T 75ab638b1a5d43e98971b5ac35b225ef P b144b2b9d65b4415a271af890cd12357: stopping tablet replica
I20260810 06:42:33.964577 1373 raft_consensus.cc:2229] T 75ab638b1a5d43e98971b5ac35b225ef P b144b2b9d65b4415a271af890cd12357 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:42:33.965042 1373 raft_consensus.cc:2258] T 75ab638b1a5d43e98971b5ac35b225ef P b144b2b9d65b4415a271af890cd12357 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:42:33.965885 1326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:33.949185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54080 (local address 127.31.59.193:45133)
0810 06:42:33.949360 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:33.949366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:33.955050 (+ 5684us) server_negotiation.cc:407] Connection header received
0810 06:42:33.955228 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:33.955234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:33.955336 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:33.955448 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:33.956132 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.957298 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.959163 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:33.959830 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:33.959925 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:33.964067 (+ 4142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:33.964080 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:33.964097 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:33.964839 (+ 742us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:33.964900 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:33.964933 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:33.965033 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:33.965408 (+ 375us) server_negotiation.cc:299] Negotiation successful
0810 06:42:33.965527 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:42:33.965888 1374 tablet_replica.cc:330] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145: stopping tablet replica
I20260810 06:42:33.966266 1374 raft_consensus.cc:2229] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:42:33.966954 1374 raft_consensus.cc:2258] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:42:33.967299 827 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:33.968788 1375 ts_tablet_manager.cc:1918] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:42:33.968950 1375 log.cc:1196] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-2/wal/wals/75ab638b1a5d43e98971b5ac35b225ef
I20260810 06:42:33.969468 1375 ts_tablet_manager.cc:1939] T 75ab638b1a5d43e98971b5ac35b225ef P e18f7d5bd11e4c139786e4f70e97e0a4: Deleting consensus metadata
I20260810 06:42:33.969851 1374 ts_tablet_manager.cc:1905] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:42:33.971585 512 catalog_manager.cc:4841] TS e18f7d5bd11e4c139786e4f70e97e0a4 (127.31.59.195:37321): tablet 75ab638b1a5d43e98971b5ac35b225ef (table test-table [id=db8a1508c5de470691999074e8df8eef]) successfully deleted
I20260810 06:42:33.976140 1374 ts_tablet_manager.cc:1918] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:42:33.976310 1374 log.cc:1196] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145: Deleting WAL directory at /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343831713822-31983-0/minicluster-data/ts-1/wal/wals/75ab638b1a5d43e98971b5ac35b225ef
I20260810 06:42:33.976858 1374 ts_tablet_manager.cc:1939] T 75ab638b1a5d43e98971b5ac35b225ef P 74eac40dcb43433f816f89470c28b145: Deleting consensus metadata
I20260810 06:42:33.978690 511 catalog_manager.cc:4841] TS 74eac40dcb43433f816f89470c28b145 (127.31.59.194:36279): tablet 75ab638b1a5d43e98971b5ac35b225ef (table test-table [id=db8a1508c5de470691999074e8df8eef]) successfully deleted
I20260810 06:42:34.078859 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 627
I20260810 06:42:34.101485 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 838
W20260810 06:42:34.101603 512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.59.193:45133 (error 108)
I20260810 06:42:34.103960 1380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:34.103149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45133 (local address 127.0.0.1:54084)
0810 06:42:34.103578 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:34.103754 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:45133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":203,"threads_started":1}
W20260810 06:42:34.104234 512 catalog_manager.cc:4584] TS b144b2b9d65b4415a271af890cd12357 (127.31.59.193:45133): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 75ab638b1a5d43e98971b5ac35b225ef: Network error: Client connection negotiation failed: client connection to 127.31.59.193:45133: connect: Connection refused (error 111)
I20260810 06:42:34.108935 1319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:34.102342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39252 (local address 127.31.59.194:36279)
0810 06:42:34.102526 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:34.102532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:34.102554 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:42:34.102731 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:34.102737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:34.102840 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:34.102946 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:34.103558 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:34.104624 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:34.105905 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:34.106402 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:34.106518 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:34.107586 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:34.107595 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:34.107610 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:34.108210 (+ 600us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:34.108262 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:34.108281 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:34.108374 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:34.108521 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 06:42:34.108617 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:42:34.109676 1048 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:34.157732 1380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:34.157263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45133 (local address 127.0.0.1:54086)
0810 06:42:34.157479 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:34.157577 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:45133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260810 06:42:34.158829 1380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:34.158433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45133 (local address 127.0.0.1:54100)
0810 06:42:34.158605 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:34.158684 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:45133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 06:42:34.219607 1380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:34.219025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45133 (local address 127.0.0.1:54104)
0810 06:42:34.219281 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:34.219405 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:45133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 06:42:34.220777 1380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:34.220404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45133 (local address 127.0.0.1:54114)
0810 06:42:34.220569 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:34.220641 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:45133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 06:42:34.221441 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 838
I20260810 06:42:34.246832 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 1063
I20260810 06:42:34.253975 1322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:34.247559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51982 (local address 127.31.59.195:37321)
0810 06:42:34.247712 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:34.247717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:34.247771 (+ 54us) server_negotiation.cc:407] Connection header received
0810 06:42:34.247908 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:34.247912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:34.247995 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:34.248085 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:34.248578 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:34.249516 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:34.250952 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:34.251379 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:34.251521 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:34.252431 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:34.252441 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:34.252457 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:34.253164 (+ 707us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:34.253249 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:34.253272 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:34.253370 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:34.253538 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 06:42:34.253654 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:42:34.254626 1275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:34.301967 1380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:34.301355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45133 (local address 127.0.0.1:54124)
0810 06:42:34.301662 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:34.301782 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:45133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260810 06:42:34.303144 1380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:34.302697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.193:45133 (local address 127.0.0.1:54130)
0810 06:42:34.302894 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 06:42:34.302983 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.193:45133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260810 06:42:34.341212 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 1063
I20260810 06:42:34.364580 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 480
I20260810 06:42:34.365700 582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:34.473404 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 480
2026-08-10T06:42:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2970 ms)
[----------] 6 tests from EncryptionPolicyTest (22277 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:42:34 dist-test-slave-4tdj krb5kdc[1390](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:42:34 dist-test-slave-4tdj krb5kdc[1390](info): set up 2 sockets
Aug 10 06:42:34 dist-test-slave-4tdj krb5kdc[1390](info): commencing operation
krb5kdc: starting...
W20260810 06:42:35.758391 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.006s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:42:35 dist-test-slave-4tdj krb5kdc[1390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:42:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:42:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:42:36.026070 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45675
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:36.354720 1416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:36.355000 1416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:36.355134 1416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:36.362077 1416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:42:36.362191 1416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:36.362269 1416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:36.362380 1416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:42:36.362432 1416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:42:36.371850 1416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45675
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:36.372983 1416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:36.374203 1416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:36.385356 1421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:36.385437 1422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:36.385414 1424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:36.386459 1416 server_base.cc:1034] running on GCE node
I20260810 06:42:36.387164 1416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:36.388424 1416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:36.389629 1416 hybrid_clock.cc:648] HybridClock initialized: now 1786344156389588 us; error 32 us; skew 500 ppm
I20260810 06:42:36.389967 1416 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:36 dist-test-slave-4tdj krb5kdc[1390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344156, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:36.392943 1416 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:42:36.394265 1416 webserver.cc:460] Webserver started at http://127.0.0.1:42777/ using document root <none> and password file <none>
I20260810 06:42:36.394891 1416 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:36.394987 1416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:36.395221 1416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:36.397967 1416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "cd6c788e23594628ae4dfc16929c714e"
format_stamp: "Formatted at 2026-08-10 06:42:36 on dist-test-slave-4tdj"
server_key: "e6facb119e567ccdce3c555b6301b946"
server_key_iv: "e3c8731d1c6334b821f9eb92e1b4cba6"
server_key_version: "encryptionkey@0"
I20260810 06:42:36.398712 1416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "cd6c788e23594628ae4dfc16929c714e"
format_stamp: "Formatted at 2026-08-10 06:42:36 on dist-test-slave-4tdj"
server_key: "e6facb119e567ccdce3c555b6301b946"
server_key_iv: "e3c8731d1c6334b821f9eb92e1b4cba6"
server_key_version: "encryptionkey@0"
I20260810 06:42:36.403470 1416 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260810 06:42:36.406787 1435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:36.407985 1416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:42:36.408135 1416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "cd6c788e23594628ae4dfc16929c714e"
format_stamp: "Formatted at 2026-08-10 06:42:36 on dist-test-slave-4tdj"
server_key: "e6facb119e567ccdce3c555b6301b946"
server_key_iv: "e3c8731d1c6334b821f9eb92e1b4cba6"
server_key_version: "encryptionkey@0"
I20260810 06:42:36.408301 1416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:36.435184 1416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:36.439071 1416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:36.439327 1416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:36.456496 1416 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45675
I20260810 06:42:36.456563 1535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45675 every 8 connection(s)
I20260810 06:42:36.457813 1416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:42:36.461548 1537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:36.462461 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 1416
I20260810 06:42:36.462694 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:42:36.463340 31983 external_mini_cluster.cc:1428] Setting key ccd0e13bb47c56e7e4167f71492b936c
I20260810 06:42:36.476032 1537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e: Bootstrap starting.
Aug 10 06:42:36 dist-test-slave-4tdj krb5kdc[1390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:42:36.479311 1537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:36.480712 1537 log.cc:824] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:36.483969 1537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e: No bootstrap required, opened a new log
I20260810 06:42:36.487172 1541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:36.465290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35579 (local address 127.0.0.1:45675)
0810 06:42:36.466140 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:36.466156 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:36.466214 (+ 58us) server_negotiation.cc:407] Connection header received
0810 06:42:36.468225 (+ 2011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:36.468255 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:36.468766 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:36.469232 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:36.470529 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:36.472512 (+ 1983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:36.473753 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:36.474832 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:36.475018 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:36.478780 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:36.478792 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:36.478810 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:36.478863 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:36.482597 (+ 3734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:36.483355 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:36.483365 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:36.483372 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:36.483603 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:36.484017 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:36.484023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:36.484026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:36.484728 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:36.485065 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:36.485425 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 06:42:36.485752 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":327,"threads_started":1}
I20260810 06:42:36.491110 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:42:36.493383 1537 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6c788e23594628ae4dfc16929c714e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45675 } }
I20260810 06:42:36.493734 1537 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:36.493847 1537 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6c788e23594628ae4dfc16929c714e, State: Initialized, Role: FOLLOWER
I20260810 06:42:36.494509 1537 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6c788e23594628ae4dfc16929c714e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45675 } }
I20260810 06:42:36.494751 1537 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:42:36.494894 1537 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:42:36.495057 1537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:36.498273 1537 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6c788e23594628ae4dfc16929c714e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45675 } }
I20260810 06:42:36.498852 1537 leader_election.cc:302] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd6c788e23594628ae4dfc16929c714e; no voters:
I20260810 06:42:36.499527 1537 leader_election.cc:288] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:42:36.499643 1553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:36.500717 1553 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e [term 1 LEADER]: Becoming Leader. State: Replica: cd6c788e23594628ae4dfc16929c714e, State: Running, Role: LEADER
I20260810 06:42:36.501336 1553 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6c788e23594628ae4dfc16929c714e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45675 } }
I20260810 06:42:36.501847 1537 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:42:36.504223 1555 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd6c788e23594628ae4dfc16929c714e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6c788e23594628ae4dfc16929c714e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45675 } } }
I20260810 06:42:36.504693 1555 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:36.509426 1556 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd6c788e23594628ae4dfc16929c714e. Latest consensus state: current_term: 1 leader_uuid: "cd6c788e23594628ae4dfc16929c714e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6c788e23594628ae4dfc16929c714e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45675 } } }
I20260810 06:42:36.509795 1556 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:36.511595 1541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:36.495530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57951 (local address 127.0.0.1:45675)
0810 06:42:36.495720 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:36.495727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:36.496211 (+ 484us) server_negotiation.cc:407] Connection header received
0810 06:42:36.496384 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:36.496388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:36.496447 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:36.496522 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:36.498510 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:36.499318 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:36.500935 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:36.501461 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:36.501598 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:36.504251 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:36.504257 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:36.504262 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:36.504295 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:36.507938 (+ 3643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:36.508930 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:36.508936 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:36.508940 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:36.509105 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:36.509722 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:36.509728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:36.509731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:36.510208 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:36.510553 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:36.510960 (+ 407us) server_negotiation.cc:299] Negotiation successful
0810 06:42:36.511161 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:42:36.513833 1560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:42:36.518865 1560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:42:36.527299 1560 catalog_manager.cc:1347] Generated new cluster ID: a0984245f70a4337b51d723f7599fc38
I20260810 06:42:36.527411 1560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:42:36.540338 1560 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:42:36.542047 1560 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:42:36.550977 1560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd6c788e23594628ae4dfc16929c714e: Generated new TSK 0
I20260810 06:42:36.551731 1560 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:42:36.564179 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 1416
I20260810 06:42:36.566282 1531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:36.654433 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 1416
W20260810 06:42:36.677752 1388 connection.cc:535] client connection to 127.0.0.1:45675 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45675): Connection reset by peer (error 104)
2026-08-10T06:42:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2175 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:42:36 dist-test-slave-4tdj krb5kdc[1588](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:42:36 dist-test-slave-4tdj krb5kdc[1588](info): set up 2 sockets
Aug 10 06:42:36 dist-test-slave-4tdj krb5kdc[1588](info): commencing operation
krb5kdc: starting...
W20260810 06:42:37.880128 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:42:38 dist-test-slave-4tdj krb5kdc[1588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:42:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:42:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:42:38.161064 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46531
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:38.493180 1610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:38.493472 1610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:38.493633 1610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:38.500808 1610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:42:38.500928 1610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:38.501008 1610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:38.501092 1610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:42:38.501152 1610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:42:38.510892 1610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46531
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:38.512195 1610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:38.513491 1610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:38.524696 1621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:38.524741 1627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:38.524785 1622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:38.525877 1610 server_base.cc:1034] running on GCE node
I20260810 06:42:38.526641 1610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:38.527983 1610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:38.529199 1610 hybrid_clock.cc:648] HybridClock initialized: now 1786344158529140 us; error 50 us; skew 500 ppm
I20260810 06:42:38.529524 1610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:38 dist-test-slave-4tdj krb5kdc[1588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344158, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:38.532671 1610 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:42:38.534116 1610 webserver.cc:460] Webserver started at http://127.0.0.1:41423/ using document root <none> and password file <none>
I20260810 06:42:38.534797 1610 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:38.534914 1610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:38.535197 1610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:38.538160 1610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "a3e7b4bad1734f9a9bfbe22830db5e0d"
format_stamp: "Formatted at 2026-08-10 06:42:38 on dist-test-slave-4tdj"
server_key: "a350a192a94b26ef5bbd5084da772ee5"
server_key_iv: "77a895784393f0d6fdf24ec634f1758f"
server_key_version: "encryptionkey@0"
I20260810 06:42:38.538962 1610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "a3e7b4bad1734f9a9bfbe22830db5e0d"
format_stamp: "Formatted at 2026-08-10 06:42:38 on dist-test-slave-4tdj"
server_key: "a350a192a94b26ef5bbd5084da772ee5"
server_key_iv: "77a895784393f0d6fdf24ec634f1758f"
server_key_version: "encryptionkey@0"
I20260810 06:42:38.544099 1610 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:42:38.547463 1642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:38.548784 1610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260810 06:42:38.548942 1610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "a3e7b4bad1734f9a9bfbe22830db5e0d"
format_stamp: "Formatted at 2026-08-10 06:42:38 on dist-test-slave-4tdj"
server_key: "a350a192a94b26ef5bbd5084da772ee5"
server_key_iv: "77a895784393f0d6fdf24ec634f1758f"
server_key_version: "encryptionkey@0"
I20260810 06:42:38.549124 1610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:38.590076 1610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:38.594146 1610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:38.594434 1610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:38.611706 1610 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46531
I20260810 06:42:38.611743 1727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46531 every 8 connection(s)
I20260810 06:42:38.613003 1610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:42:38.616622 1728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:38.619520 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 1610
I20260810 06:42:38.619702 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:42:38.620216 31983 external_mini_cluster.cc:1428] Setting key 897a8bb883610cc571977aaef05d04cf
I20260810 06:42:38.627914 1728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d: Bootstrap starting.
I20260810 06:42:38.630980 1728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:38.632083 1728 log.cc:824] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:38.632933 1731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:38.622045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38635 (local address 127.0.0.1:46531)
0810 06:42:38.622879 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:38.622890 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:38.622929 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:42:38.624583 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:38.624604 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:38.624979 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:38.625339 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:38.625871 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:38.627220 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:38.628581 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:38.629234 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:38.629400 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:38.630633 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:38.630640 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:38.630653 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:38.631187 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:38.631268 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:38.631294 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:38.631447 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:38.631701 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 06:42:38.631843 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":409,"threads_started":1}
I20260810 06:42:38.635102 1728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d: No bootstrap required, opened a new log
I20260810 06:42:38.636593 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:42:38.642771 1728 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3e7b4bad1734f9a9bfbe22830db5e0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46531 } }
I20260810 06:42:38.643132 1728 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:38.643219 1728 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3e7b4bad1734f9a9bfbe22830db5e0d, State: Initialized, Role: FOLLOWER
I20260810 06:42:38.643767 1728 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3e7b4bad1734f9a9bfbe22830db5e0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46531 } }
I20260810 06:42:38.643992 1728 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:42:38.644114 1728 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:42:38.644277 1728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:38.646579 1731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:38.640612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38321 (local address 127.0.0.1:46531)
0810 06:42:38.640746 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:38.640750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:38.641232 (+ 482us) server_negotiation.cc:407] Connection header received
0810 06:42:38.641404 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:38.641411 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:38.641469 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:38.641574 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:38.642033 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:38.642786 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:38.644238 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:38.644702 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:38.644823 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:38.645886 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:38.645891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:38.645894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:38.645910 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:38.645943 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:38.645958 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:38.646042 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:38.646244 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 06:42:38.646321 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:42:38.646776 1728 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3e7b4bad1734f9a9bfbe22830db5e0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46531 } }
I20260810 06:42:38.647266 1728 leader_election.cc:302] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3e7b4bad1734f9a9bfbe22830db5e0d; no voters:
I20260810 06:42:38.647881 1728 leader_election.cc:288] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:42:38.648016 1741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:38.648641 1741 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d [term 1 LEADER]: Becoming Leader. State: Replica: a3e7b4bad1734f9a9bfbe22830db5e0d, State: Running, Role: LEADER
I20260810 06:42:38.649186 1741 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3e7b4bad1734f9a9bfbe22830db5e0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46531 } }
I20260810 06:42:38.650085 1728 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:42:38.653017 1742 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3e7b4bad1734f9a9bfbe22830db5e0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3e7b4bad1734f9a9bfbe22830db5e0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46531 } } }
I20260810 06:42:38.652056 1743 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3e7b4bad1734f9a9bfbe22830db5e0d. Latest consensus state: current_term: 1 leader_uuid: "a3e7b4bad1734f9a9bfbe22830db5e0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3e7b4bad1734f9a9bfbe22830db5e0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46531 } } }
I20260810 06:42:38.653434 1742 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:38.653452 1743 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:38.655167 1757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:42:38.659878 1757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:42:38.668278 1757 catalog_manager.cc:1347] Generated new cluster ID: ee323ff16ebe42ad984a3d6f13714742
I20260810 06:42:38.668378 1757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:42:38.689226 1757 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:42:38.690449 1757 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:42:38.701629 1757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3e7b4bad1734f9a9bfbe22830db5e0d: Generated new TSK 0
I20260810 06:42:38.702657 1757 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:42:38.719472 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 1610
W20260810 06:42:38.720527 1721 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38635
W20260810 06:42:38.721287 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 1610: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:42:38.721457 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 1610
W20260810 06:42:38.744707 1585 connection.cc:535] client connection to 127.0.0.1:46531 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46531): Connection reset by peer (error 104)
2026-08-10T06:42:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2066 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:42:38 dist-test-slave-4tdj krb5kdc[1778](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:42:38 dist-test-slave-4tdj krb5kdc[1778](info): set up 2 sockets
Aug 10 06:42:38 dist-test-slave-4tdj krb5kdc[1778](info): commencing operation
krb5kdc: starting...
W20260810 06:42:39.955435 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:42:40 dist-test-slave-4tdj krb5kdc[1778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:42:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:42:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:42:40.233672 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33707
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:40.561208 1799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:40.561506 1799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:40.561657 1799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:40.568840 1799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:42:40.568971 1799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:40.569058 1799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:40.569141 1799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:42:40.569201 1799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:42:40.578962 1799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33707
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:40.580219 1799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:40.581504 1799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:40.593045 1810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:40.593152 1804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:40.593124 1807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:40.593883 1799 server_base.cc:1034] running on GCE node
I20260810 06:42:40.594643 1799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:40.595920 1799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:40.597117 1799 hybrid_clock.cc:648] HybridClock initialized: now 1786344160597077 us; error 31 us; skew 500 ppm
I20260810 06:42:40.597442 1799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:40 dist-test-slave-4tdj krb5kdc[1778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344160, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:40.604665 1799 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:42:40.606084 1799 webserver.cc:460] Webserver started at http://127.0.0.1:33687/ using document root <none> and password file <none>
I20260810 06:42:40.606758 1799 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:40.606878 1799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:40.607148 1799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:40.609987 1799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "7f2eaecdcdde4cf696ed9867c2b6956f"
format_stamp: "Formatted at 2026-08-10 06:42:40 on dist-test-slave-4tdj"
server_key: "b43831c89cc95ac4d37a7330ec381af3"
server_key_iv: "1944d1a31f660b40dfc2b5ef44a03527"
server_key_version: "encryptionkey@0"
I20260810 06:42:40.610759 1799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "7f2eaecdcdde4cf696ed9867c2b6956f"
format_stamp: "Formatted at 2026-08-10 06:42:40 on dist-test-slave-4tdj"
server_key: "b43831c89cc95ac4d37a7330ec381af3"
server_key_iv: "1944d1a31f660b40dfc2b5ef44a03527"
server_key_version: "encryptionkey@0"
I20260810 06:42:40.615638 1799 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260810 06:42:40.618932 1818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:40.620226 1799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:42:40.620368 1799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "7f2eaecdcdde4cf696ed9867c2b6956f"
format_stamp: "Formatted at 2026-08-10 06:42:40 on dist-test-slave-4tdj"
server_key: "b43831c89cc95ac4d37a7330ec381af3"
server_key_iv: "1944d1a31f660b40dfc2b5ef44a03527"
server_key_version: "encryptionkey@0"
I20260810 06:42:40.620532 1799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:40.626050 1799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:40.630008 1799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:40.630251 1799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:40.646411 1799 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33707
I20260810 06:42:40.646456 1910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33707 every 8 connection(s)
I20260810 06:42:40.647701 1799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:42:40.650025 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 1799
I20260810 06:42:40.650204 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:42:40.650774 31983 external_mini_cluster.cc:1428] Setting key 9e121be2b6e370eef950591ac61230d9
I20260810 06:42:40.651772 1912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:40.658077 1917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:40.652521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40849 (local address 127.0.0.1:33707)
0810 06:42:40.653262 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:40.653274 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:40.653316 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:42:40.654860 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:40.654881 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:40.655213 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:40.655539 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:40.656556 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:40.656563 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:40.656576 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:40.656592 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:40.656680 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:40.656702 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:40.656887 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:40.656994 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 06:42:40.657151 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":339,"threads_started":1}
I20260810 06:42:40.661446 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:42:40.665005 1912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f: Bootstrap starting.
I20260810 06:42:40.668849 1912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:40.668892 1917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:40.666734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36427 (local address 127.0.0.1:33707)
0810 06:42:40.666911 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:40.666916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:40.667187 (+ 271us) server_negotiation.cc:407] Connection header received
0810 06:42:40.667350 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:40.667353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:40.667400 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:40.667474 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:40.668310 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:40.668313 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:40.668316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:40.668330 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:40.668368 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:40.668381 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:40.668461 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:40.668597 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 06:42:40.668638 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 06:42:40.669901 1912 log.cc:824] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:40.672366 1912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f: No bootstrap required, opened a new log
I20260810 06:42:40.679224 1912 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2eaecdcdde4cf696ed9867c2b6956f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33707 } }
I20260810 06:42:40.679587 1912 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:40.679687 1912 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f2eaecdcdde4cf696ed9867c2b6956f, State: Initialized, Role: FOLLOWER
I20260810 06:42:40.680214 1912 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2eaecdcdde4cf696ed9867c2b6956f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33707 } }
I20260810 06:42:40.680426 1912 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:42:40.680559 1912 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:42:40.680712 1912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:40.682827 1912 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2eaecdcdde4cf696ed9867c2b6956f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33707 } }
I20260810 06:42:40.683276 1912 leader_election.cc:302] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f2eaecdcdde4cf696ed9867c2b6956f; no voters:
I20260810 06:42:40.683816 1912 leader_election.cc:288] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:42:40.683933 1929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:40.684950 1929 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f [term 1 LEADER]: Becoming Leader. State: Replica: 7f2eaecdcdde4cf696ed9867c2b6956f, State: Running, Role: LEADER
I20260810 06:42:40.685545 1929 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2eaecdcdde4cf696ed9867c2b6956f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33707 } }
I20260810 06:42:40.685783 1912 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:42:40.688089 1932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f2eaecdcdde4cf696ed9867c2b6956f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2eaecdcdde4cf696ed9867c2b6956f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33707 } } }
I20260810 06:42:40.688496 1932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:40.688407 1933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f2eaecdcdde4cf696ed9867c2b6956f. Latest consensus state: current_term: 1 leader_uuid: "7f2eaecdcdde4cf696ed9867c2b6956f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2eaecdcdde4cf696ed9867c2b6956f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33707 } } }
I20260810 06:42:40.688660 1933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:40.690375 1948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:42:40.695076 1948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:42:40.704440 1948 catalog_manager.cc:1347] Generated new cluster ID: dc6194e7cea94674b1359b5a77260cae
I20260810 06:42:40.704542 1948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:42:40.718044 1948 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:42:40.718434 1948 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:42:40.725376 1948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f2eaecdcdde4cf696ed9867c2b6956f: Generated new TSK 0
I20260810 06:42:40.726063 1948 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:42:40.732367 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 1799
W20260810 06:42:40.733331 1901 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40849
W20260810 06:42:40.734174 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 1799: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:42:40.734393 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 1799
2026-08-10T06:42:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2014 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:42:40 dist-test-slave-4tdj krb5kdc[1973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:42:40 dist-test-slave-4tdj krb5kdc[1973](info): set up 2 sockets
Aug 10 06:42:40 dist-test-slave-4tdj krb5kdc[1973](info): commencing operation
krb5kdc: starting...
W20260810 06:42:41.971643 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:42:42 dist-test-slave-4tdj krb5kdc[1973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:42:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:42:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:42:42.251350 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33655
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:42.576848 1989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:42.577119 1989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:42.577219 1989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:42.584136 1989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:42:42.584241 1989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:42.584295 1989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:42.584354 1989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:42:42.584399 1989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:42:42.594391 1989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:42.595588 1989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:42.596822 1989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:42.608057 1995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:42.608178 1996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:42.607965 2000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:42.609174 1989 server_base.cc:1034] running on GCE node
I20260810 06:42:42.610100 1989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:42.611517 1989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:42.612711 1989 hybrid_clock.cc:648] HybridClock initialized: now 1786344162612668 us; error 38 us; skew 500 ppm
I20260810 06:42:42.613032 1989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:42 dist-test-slave-4tdj krb5kdc[1973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344162, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:42.615970 1989 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:42:42.617358 1989 webserver.cc:460] Webserver started at http://127.0.0.1:38765/ using document root <none> and password file <none>
I20260810 06:42:42.617964 1989 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:42.618069 1989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:42.618351 1989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:42.621398 1989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "66e417935fc1474aa81cd68023efc94e"
format_stamp: "Formatted at 2026-08-10 06:42:42 on dist-test-slave-4tdj"
server_key: "d67d2d9977a986f042e7b4e198abf6fc"
server_key_iv: "c36bea9a7781790c75001751a6d617a9"
server_key_version: "encryptionkey@0"
I20260810 06:42:42.622108 1989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "66e417935fc1474aa81cd68023efc94e"
format_stamp: "Formatted at 2026-08-10 06:42:42 on dist-test-slave-4tdj"
server_key: "d67d2d9977a986f042e7b4e198abf6fc"
server_key_iv: "c36bea9a7781790c75001751a6d617a9"
server_key_version: "encryptionkey@0"
I20260810 06:42:42.627027 1989 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260810 06:42:42.630198 2010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:42.631443 1989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:42:42.631608 1989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "66e417935fc1474aa81cd68023efc94e"
format_stamp: "Formatted at 2026-08-10 06:42:42 on dist-test-slave-4tdj"
server_key: "d67d2d9977a986f042e7b4e198abf6fc"
server_key_iv: "c36bea9a7781790c75001751a6d617a9"
server_key_version: "encryptionkey@0"
I20260810 06:42:42.631785 1989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:42.645979 1989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:42.649781 1989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:42.650034 1989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:42.667452 1989 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33655
I20260810 06:42:42.667474 2105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33655 every 8 connection(s)
I20260810 06:42:42.668797 1989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:42:42.672425 2106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:42.677369 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 1989
I20260810 06:42:42.677558 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:42:42.678048 31983 external_mini_cluster.cc:1428] Setting key fc5707b35d83acda68cd9ecbb281dcd6
I20260810 06:42:42.683271 2106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e: Bootstrap starting.
I20260810 06:42:42.686825 2106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:42.688031 2106 log.cc:824] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:42.690830 2106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e: No bootstrap required, opened a new log
Aug 10 06:42:42 dist-test-slave-4tdj krb5kdc[1973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:42:42.699780 2106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e417935fc1474aa81cd68023efc94e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33655 } }
I20260810 06:42:42.700133 2106 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:42.700227 2106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e417935fc1474aa81cd68023efc94e, State: Initialized, Role: FOLLOWER
I20260810 06:42:42.700397 2109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:42.679728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49213 (local address 127.0.0.1:33655)
0810 06:42:42.680527 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:42.680537 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:42.680585 (+ 48us) server_negotiation.cc:407] Connection header received
0810 06:42:42.682251 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:42.682274 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:42.682733 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:42.683041 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:42.684799 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:42.686109 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:42.687883 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:42.688652 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:42.688847 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:42.692971 (+ 4124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:42.692978 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:42.692992 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:42.693029 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:42.696183 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:42.697164 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:42.697175 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:42.697182 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:42.697358 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:42.697753 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:42.697756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:42.697761 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:42.698354 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:42.698608 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:42.699002 (+ 394us) server_negotiation.cc:299] Negotiation successful
0810 06:42:42.699338 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":368,"threads_started":1}
I20260810 06:42:42.700842 2106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e417935fc1474aa81cd68023efc94e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33655 } }
I20260810 06:42:42.701095 2106 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:42:42.701225 2106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:42:42.701398 2106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:42.703873 2106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e417935fc1474aa81cd68023efc94e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33655 } }
I20260810 06:42:42.703974 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:42:42.704423 2106 leader_election.cc:302] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66e417935fc1474aa81cd68023efc94e; no voters:
I20260810 06:42:42.705022 2106 leader_election.cc:288] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:42:42.705144 2112 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:42.706146 2112 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e [term 1 LEADER]: Becoming Leader. State: Replica: 66e417935fc1474aa81cd68023efc94e, State: Running, Role: LEADER
I20260810 06:42:42.706679 2112 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e417935fc1474aa81cd68023efc94e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33655 } }
I20260810 06:42:42.707139 2106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:42:42.709462 2115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66e417935fc1474aa81cd68023efc94e. Latest consensus state: current_term: 1 leader_uuid: "66e417935fc1474aa81cd68023efc94e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e417935fc1474aa81cd68023efc94e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33655 } } }
I20260810 06:42:42.709697 2114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66e417935fc1474aa81cd68023efc94e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e417935fc1474aa81cd68023efc94e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33655 } } }
I20260810 06:42:42.709920 2115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:42.709941 2114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:42.718708 2131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:42:42.723511 2131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:42:42.734565 2109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:42.710171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46099 (local address 127.0.0.1:33655)
0810 06:42:42.710379 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:42.710384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:42.717918 (+ 7534us) server_negotiation.cc:407] Connection header received
0810 06:42:42.718202 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:42.718208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:42.718317 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:42.718389 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:42.720434 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:42.721532 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:42.723386 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:42.724052 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:42.724192 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:42.727349 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:42.727356 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:42.727361 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:42.727397 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:42.731078 (+ 3681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:42.731976 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:42.731982 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:42.731985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:42.732142 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:42.732610 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:42.732615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:42.732619 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:42.733083 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:42.733318 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:42.733920 (+ 602us) server_negotiation.cc:299] Negotiation successful
0810 06:42:42.734148 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:42:42.740662 2131 catalog_manager.cc:1347] Generated new cluster ID: ec040d9160784ea9b1dc12883dfefca3
I20260810 06:42:42.740764 2131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:42:42.755622 2131 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:42:42.757378 2131 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:42:42.768754 2131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66e417935fc1474aa81cd68023efc94e: Generated new TSK 0
I20260810 06:42:42.769740 2131 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:42:42.776139 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 1989
I20260810 06:42:42.777204 2098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:42.864773 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 1989
2026-08-10T06:42:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2130 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:42:42 dist-test-slave-4tdj krb5kdc[2151](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:42:42 dist-test-slave-4tdj krb5kdc[2151](info): set up 2 sockets
Aug 10 06:42:42 dist-test-slave-4tdj krb5kdc[2151](info): commencing operation
krb5kdc: starting...
W20260810 06:42:44.105566 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.006s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:42:44 dist-test-slave-4tdj krb5kdc[2151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:42:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:42:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:42:44.374009 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36143
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:44.705971 2180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:44.706250 2180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:44.706414 2180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:44.713552 2180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:42:44.713666 2180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:44.713747 2180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:44.713840 2180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:42:44.713903 2180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:42:44.723615 2180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:44.724843 2180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:44.726212 2180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:44.738459 2190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:44.738533 2180 server_base.cc:1034] running on GCE node
W20260810 06:42:44.738535 2191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:44.738583 2193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:44.739655 2180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:44.741060 2180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:44.742265 2180 hybrid_clock.cc:648] HybridClock initialized: now 1786344164742216 us; error 39 us; skew 500 ppm
I20260810 06:42:44.742626 2180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:44 dist-test-slave-4tdj krb5kdc[2151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344164, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:44.745615 2180 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:42:44.747145 2180 webserver.cc:460] Webserver started at http://127.0.0.1:46367/ using document root <none> and password file <none>
I20260810 06:42:44.747788 2180 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:44.747888 2180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:44.748178 2180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:44.751101 2180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "f9dcfdff29d84f25a060ce8bd68ee255"
format_stamp: "Formatted at 2026-08-10 06:42:44 on dist-test-slave-4tdj"
server_key: "4577884766f9222fe9e29fab81204d85"
server_key_iv: "54411c91357852b44eb6253b7e940e0d"
server_key_version: "encryptionkey@0"
I20260810 06:42:44.751830 2180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "f9dcfdff29d84f25a060ce8bd68ee255"
format_stamp: "Formatted at 2026-08-10 06:42:44 on dist-test-slave-4tdj"
server_key: "4577884766f9222fe9e29fab81204d85"
server_key_iv: "54411c91357852b44eb6253b7e940e0d"
server_key_version: "encryptionkey@0"
I20260810 06:42:44.756783 2180 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:42:44.760053 2206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:44.761291 2180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:42:44.761444 2180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "f9dcfdff29d84f25a060ce8bd68ee255"
format_stamp: "Formatted at 2026-08-10 06:42:44 on dist-test-slave-4tdj"
server_key: "4577884766f9222fe9e29fab81204d85"
server_key_iv: "54411c91357852b44eb6253b7e940e0d"
server_key_version: "encryptionkey@0"
I20260810 06:42:44.761626 2180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:44.773891 2180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:44.777544 2180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:44.777783 2180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:44.794895 2180 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36143
I20260810 06:42:44.794927 2296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36143 every 8 connection(s)
I20260810 06:42:44.796192 2180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:42:44.796816 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 2180
I20260810 06:42:44.797014 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:42:44.797520 31983 external_mini_cluster.cc:1428] Setting key 6f5da26d4cd30805c3c8b581ab0a67af
I20260810 06:42:44.800354 2297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:44.810515 2302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:44.799270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60365 (local address 127.0.0.1:36143)
0810 06:42:44.800024 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:44.800035 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:44.800076 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:42:44.801701 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:44.801723 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:44.802074 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:44.802415 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:44.802999 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:44.804294 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:44.806182 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:44.806964 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:44.807107 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:44.808279 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:44.808286 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:44.808299 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:44.808838 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:44.808908 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:44.808935 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:44.809020 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:44.809249 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 06:42:44.809498 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":346,"threads_started":1}
I20260810 06:42:44.812124 2297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255: Bootstrap starting.
I20260810 06:42:44.813810 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:42:44.815585 2297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:44.816818 2297 log.cc:824] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:44.819800 2297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255: No bootstrap required, opened a new log
I20260810 06:42:44.825034 2302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:44.818845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37721 (local address 127.0.0.1:36143)
0810 06:42:44.819011 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:44.819017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:44.819497 (+ 480us) server_negotiation.cc:407] Connection header received
0810 06:42:44.819641 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:44.819646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:44.819714 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:44.819831 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:44.820287 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:44.821302 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:44.822618 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:44.823312 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:44.823440 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:44.824323 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:44.824329 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:44.824335 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:44.824354 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:44.824400 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:44.824419 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:44.824513 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:44.824617 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 06:42:44.824706 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:42:44.828182 2297 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9dcfdff29d84f25a060ce8bd68ee255" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36143 } }
I20260810 06:42:44.828503 2297 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:44.828594 2297 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9dcfdff29d84f25a060ce8bd68ee255, State: Initialized, Role: FOLLOWER
I20260810 06:42:44.829113 2297 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9dcfdff29d84f25a060ce8bd68ee255" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36143 } }
I20260810 06:42:44.829334 2297 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:42:44.829463 2297 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:42:44.829610 2297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:44.831820 2297 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9dcfdff29d84f25a060ce8bd68ee255" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36143 } }
I20260810 06:42:44.832342 2297 leader_election.cc:302] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9dcfdff29d84f25a060ce8bd68ee255; no voters:
I20260810 06:42:44.832887 2297 leader_election.cc:288] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:42:44.832999 2319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:44.833647 2319 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255 [term 1 LEADER]: Becoming Leader. State: Replica: f9dcfdff29d84f25a060ce8bd68ee255, State: Running, Role: LEADER
I20260810 06:42:44.834129 2319 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9dcfdff29d84f25a060ce8bd68ee255" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36143 } }
I20260810 06:42:44.834829 2297 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:42:44.836838 2320 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9dcfdff29d84f25a060ce8bd68ee255" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9dcfdff29d84f25a060ce8bd68ee255" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36143 } } }
I20260810 06:42:44.837213 2320 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:44.837096 2321 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9dcfdff29d84f25a060ce8bd68ee255. Latest consensus state: current_term: 1 leader_uuid: "f9dcfdff29d84f25a060ce8bd68ee255" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9dcfdff29d84f25a060ce8bd68ee255" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36143 } } }
I20260810 06:42:44.837426 2321 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:44.839870 2332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:42:44.844812 2332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:42:44.854066 2332 catalog_manager.cc:1347] Generated new cluster ID: b703658bc89e4f7b8e81997cd8e630cd
I20260810 06:42:44.854187 2332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:42:44.867707 2332 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:42:44.869421 2332 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:42:44.878546 2332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9dcfdff29d84f25a060ce8bd68ee255: Generated new TSK 0
I20260810 06:42:44.879310 2332 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:42:44.891505 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 2180
W20260810 06:42:44.892534 2282 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60365
W20260810 06:42:44.893178 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 2180: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:42:44.893333 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 2180
2026-08-10T06:42:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2027 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:42:44 dist-test-slave-4tdj krb5kdc[2354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:42:44 dist-test-slave-4tdj krb5kdc[2354](info): set up 2 sockets
Aug 10 06:42:44 dist-test-slave-4tdj krb5kdc[2354](info): commencing operation
krb5kdc: starting...
W20260810 06:42:46.119757 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:42:46 dist-test-slave-4tdj krb5kdc[2354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:42:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:42:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:42:46.392688 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44219
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:46.716652 2370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:46.716912 2370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:46.717044 2370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:46.723933 2370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:42:46.724042 2370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:46.724119 2370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:46.724205 2370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:42:46.724263 2370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:42:46.733830 2370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:46.735100 2370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:46.736433 2370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:46.747408 2381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:46.747484 2387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:46.747521 2379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:46.748255 2370 server_base.cc:1034] running on GCE node
I20260810 06:42:46.748955 2370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:46.750242 2370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:46.751466 2370 hybrid_clock.cc:648] HybridClock initialized: now 1786344166751419 us; error 34 us; skew 500 ppm
I20260810 06:42:46.751783 2370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:46 dist-test-slave-4tdj krb5kdc[2354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344166, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:46.759107 2370 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:42:46.760465 2370 webserver.cc:460] Webserver started at http://127.0.0.1:35629/ using document root <none> and password file <none>
I20260810 06:42:46.761080 2370 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:46.761181 2370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:46.761422 2370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:46.764303 2370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "c3b8cb6ee17249d3b678cb5f27eb9d45"
format_stamp: "Formatted at 2026-08-10 06:42:46 on dist-test-slave-4tdj"
server_key: "7033c13199ebc24d9e9c40a7814e4690"
server_key_iv: "a5f34884cc4c72ae890b556a144675b8"
server_key_version: "encryptionkey@0"
I20260810 06:42:46.764987 2370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "c3b8cb6ee17249d3b678cb5f27eb9d45"
format_stamp: "Formatted at 2026-08-10 06:42:46 on dist-test-slave-4tdj"
server_key: "7033c13199ebc24d9e9c40a7814e4690"
server_key_iv: "a5f34884cc4c72ae890b556a144675b8"
server_key_version: "encryptionkey@0"
I20260810 06:42:46.770037 2370 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260810 06:42:46.773283 2396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:46.774590 2370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:42:46.774734 2370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "c3b8cb6ee17249d3b678cb5f27eb9d45"
format_stamp: "Formatted at 2026-08-10 06:42:46 on dist-test-slave-4tdj"
server_key: "7033c13199ebc24d9e9c40a7814e4690"
server_key_iv: "a5f34884cc4c72ae890b556a144675b8"
server_key_version: "encryptionkey@0"
I20260810 06:42:46.774904 2370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:46.780217 2370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:46.784029 2370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:46.784283 2370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:46.800252 2370 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44219
I20260810 06:42:46.800297 2489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44219 every 8 connection(s)
I20260810 06:42:46.801517 2370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:42:46.805090 2491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:46.806923 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 2370
I20260810 06:42:46.807082 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:42:46.807494 31983 external_mini_cluster.cc:1428] Setting key 5a19eb1bb3c1e867b4b66a8dab646cba
I20260810 06:42:46.815568 2495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:46.809261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60055 (local address 127.0.0.1:44219)
0810 06:42:46.810073 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:46.810087 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:46.810138 (+ 51us) server_negotiation.cc:407] Connection header received
0810 06:42:46.811859 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:46.811892 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:46.812312 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:46.812731 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:46.813638 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:46.813645 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:46.813662 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:46.813679 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:46.813749 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:46.813790 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:46.814001 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:46.814128 (+ 127us) server_negotiation.cc:299] Negotiation successful
0810 06:42:46.814341 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":317,"threads_started":1}
I20260810 06:42:46.818987 2491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45: Bootstrap starting.
I20260810 06:42:46.819128 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:42:46.822851 2491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:46.824210 2491 log.cc:824] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:46.826611 2495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:46.824278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40381 (local address 127.0.0.1:44219)
0810 06:42:46.824449 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:46.824454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:46.824762 (+ 308us) server_negotiation.cc:407] Connection header received
0810 06:42:46.824939 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:46.824943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:46.824996 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:46.825093 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:46.825975 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:46.825981 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:46.825986 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:46.826004 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:46.826048 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:46.826064 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:46.826158 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:46.826266 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 06:42:46.826352 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:42:46.827252 2491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45: No bootstrap required, opened a new log
I20260810 06:42:46.834817 2491 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b8cb6ee17249d3b678cb5f27eb9d45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44219 } }
I20260810 06:42:46.835171 2491 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:46.835260 2491 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b8cb6ee17249d3b678cb5f27eb9d45, State: Initialized, Role: FOLLOWER
I20260810 06:42:46.835779 2491 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b8cb6ee17249d3b678cb5f27eb9d45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44219 } }
I20260810 06:42:46.836005 2491 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:42:46.836130 2491 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:42:46.836269 2491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:46.838346 2491 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b8cb6ee17249d3b678cb5f27eb9d45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44219 } }
I20260810 06:42:46.838819 2491 leader_election.cc:302] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3b8cb6ee17249d3b678cb5f27eb9d45; no voters:
I20260810 06:42:46.839367 2491 leader_election.cc:288] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:42:46.839530 2509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:46.840143 2509 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45 [term 1 LEADER]: Becoming Leader. State: Replica: c3b8cb6ee17249d3b678cb5f27eb9d45, State: Running, Role: LEADER
I20260810 06:42:46.840734 2509 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b8cb6ee17249d3b678cb5f27eb9d45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44219 } }
I20260810 06:42:46.841424 2491 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:42:46.843353 2511 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3b8cb6ee17249d3b678cb5f27eb9d45. Latest consensus state: current_term: 1 leader_uuid: "c3b8cb6ee17249d3b678cb5f27eb9d45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b8cb6ee17249d3b678cb5f27eb9d45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44219 } } }
I20260810 06:42:46.843812 2511 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:46.843377 2510 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3b8cb6ee17249d3b678cb5f27eb9d45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b8cb6ee17249d3b678cb5f27eb9d45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44219 } } }
I20260810 06:42:46.844234 2510 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:46.845295 2518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:42:46.850340 2518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:42:46.859012 2518 catalog_manager.cc:1347] Generated new cluster ID: 8b9d5b242b0f4d3fb92b03c5fe7079a2
I20260810 06:42:46.859117 2518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:42:46.875077 2518 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:42:46.875546 2518 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:42:46.883467 2518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3b8cb6ee17249d3b678cb5f27eb9d45: Generated new TSK 0
I20260810 06:42:46.884248 2518 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:42:46.891484 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 2370
W20260810 06:42:46.892472 2484 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60055
W20260810 06:42:46.893211 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 2370: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:42:46.893353 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 2370
2026-08-10T06:42:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2001 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:42:47 dist-test-slave-4tdj krb5kdc[2545](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:42:47 dist-test-slave-4tdj krb5kdc[2545](info): set up 2 sockets
Aug 10 06:42:47 dist-test-slave-4tdj krb5kdc[2545](info): commencing operation
krb5kdc: starting...
W20260810 06:42:48.133821 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:42:48 dist-test-slave-4tdj krb5kdc[2545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:42:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:42:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:42:48.405222 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38539
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:48.738510 2568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:48.738821 2568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:48.738965 2568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:48.746184 2568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:42:48.746326 2568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:48.746412 2568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:48.746503 2568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:42:48.746564 2568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:42:48.756362 2568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:48.757660 2568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:48.758968 2568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:48.770059 2578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:48.770099 2579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:48.770138 2583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:48.771220 2568 server_base.cc:1034] running on GCE node
I20260810 06:42:48.771955 2568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:48.773288 2568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:48.774477 2568 hybrid_clock.cc:648] HybridClock initialized: now 1786344168774431 us; error 34 us; skew 500 ppm
I20260810 06:42:48.774788 2568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:48 dist-test-slave-4tdj krb5kdc[2545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344168, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:48.777735 2568 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:42:48.779140 2568 webserver.cc:460] Webserver started at http://127.0.0.1:43735/ using document root <none> and password file <none>
I20260810 06:42:48.779765 2568 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:48.779884 2568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:48.780181 2568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:48.783111 2568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "c908675bb57d4ea8823ecb2a7984b588"
format_stamp: "Formatted at 2026-08-10 06:42:48 on dist-test-slave-4tdj"
server_key: "dabbdc79d1079eba7c778bcbf5470e62"
server_key_iv: "5fe69ea41de633e76cdc927b0d856e40"
server_key_version: "encryptionkey@0"
I20260810 06:42:48.783846 2568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "c908675bb57d4ea8823ecb2a7984b588"
format_stamp: "Formatted at 2026-08-10 06:42:48 on dist-test-slave-4tdj"
server_key: "dabbdc79d1079eba7c778bcbf5470e62"
server_key_iv: "5fe69ea41de633e76cdc927b0d856e40"
server_key_version: "encryptionkey@0"
I20260810 06:42:48.788692 2568 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:42:48.791975 2596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:48.793179 2568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:42:48.793341 2568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "c908675bb57d4ea8823ecb2a7984b588"
format_stamp: "Formatted at 2026-08-10 06:42:48 on dist-test-slave-4tdj"
server_key: "dabbdc79d1079eba7c778bcbf5470e62"
server_key_iv: "5fe69ea41de633e76cdc927b0d856e40"
server_key_version: "encryptionkey@0"
I20260810 06:42:48.793520 2568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:48.815599 2568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:48.819731 2568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:48.819999 2568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:48.837101 2568 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38539
I20260810 06:42:48.837174 2706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38539 every 8 connection(s)
I20260810 06:42:48.838485 2568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:42:48.842129 2711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:48.842401 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 2568
I20260810 06:42:48.842583 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:42:48.843146 31983 external_mini_cluster.cc:1428] Setting key f091f653fb2db490565da1e1df6d2448
I20260810 06:42:48.855540 2711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588: Bootstrap starting.
Aug 10 06:42:48 dist-test-slave-4tdj krb5kdc[2545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:42:48.859347 2711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:48.860507 2711 log.cc:824] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:48.863664 2711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588: No bootstrap required, opened a new log
I20260810 06:42:48.866657 2715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:48.844979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:50749 (local address 127.0.0.1:38539)
0810 06:42:48.845819 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:48.845829 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:48.845868 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:42:48.848118 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:48.848139 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:48.848534 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:48.848860 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:48.850567 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:48.851830 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:48.853567 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:48.854217 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:48.854378 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:48.859134 (+ 4756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:48.859142 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:48.859154 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:48.859189 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:48.862431 (+ 3242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:48.863403 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:48.863410 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:48.863416 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:48.863580 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:48.864069 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:48.864072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:48.864074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:48.864573 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:48.864836 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:48.865258 (+ 422us) server_negotiation.cc:299] Negotiation successful
0810 06:42:48.865521 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":391,"threads_started":1}
I20260810 06:42:48.870522 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:42:48.873013 2711 raft_consensus.cc:348] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c908675bb57d4ea8823ecb2a7984b588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38539 } }
I20260810 06:42:48.873448 2711 raft_consensus.cc:374] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:48.873560 2711 raft_consensus.cc:729] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c908675bb57d4ea8823ecb2a7984b588, State: Initialized, Role: FOLLOWER
I20260810 06:42:48.874236 2711 consensus_queue.cc:260] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c908675bb57d4ea8823ecb2a7984b588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38539 } }
I20260810 06:42:48.874547 2711 raft_consensus.cc:388] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:42:48.874701 2711 raft_consensus.cc:482] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:42:48.874886 2711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:48.877453 2711 raft_consensus.cc:504] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c908675bb57d4ea8823ecb2a7984b588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38539 } }
I20260810 06:42:48.877910 2711 leader_election.cc:302] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c908675bb57d4ea8823ecb2a7984b588; no voters:
I20260810 06:42:48.878616 2711 leader_election.cc:288] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:42:48.878700 2727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:48.880700 2711 sys_catalog.cc:564] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:42:48.885344 2727 raft_consensus.cc:686] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588 [term 1 LEADER]: Becoming Leader. State: Replica: c908675bb57d4ea8823ecb2a7984b588, State: Running, Role: LEADER
I20260810 06:42:48.885871 2727 consensus_queue.cc:237] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c908675bb57d4ea8823ecb2a7984b588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38539 } }
I20260810 06:42:48.888823 2734 sys_catalog.cc:455] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c908675bb57d4ea8823ecb2a7984b588. Latest consensus state: current_term: 1 leader_uuid: "c908675bb57d4ea8823ecb2a7984b588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c908675bb57d4ea8823ecb2a7984b588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38539 } } }
I20260810 06:42:48.889225 2734 sys_catalog.cc:458] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:48.890017 2745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:42:48.888804 2731 sys_catalog.cc:455] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c908675bb57d4ea8823ecb2a7984b588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c908675bb57d4ea8823ecb2a7984b588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38539 } } }
I20260810 06:42:48.890930 2731 sys_catalog.cc:458] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:48.894114 2715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:48.875827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:44171 (local address 127.0.0.1:38539)
0810 06:42:48.876003 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:48.876008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:48.876381 (+ 373us) server_negotiation.cc:407] Connection header received
0810 06:42:48.876509 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:48.876514 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:48.876592 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:48.876691 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:48.878045 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:48.879337 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:48.880988 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:48.881696 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:48.881824 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:48.885462 (+ 3638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:48.885469 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:48.885473 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:48.885507 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:48.889563 (+ 4056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:48.890701 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:48.890707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:48.890711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:48.890942 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:48.891555 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:48.891560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:48.891564 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:48.892037 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:48.893344 (+ 1307us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:48.893529 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 06:42:48.893721 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:42:48.898283 2745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:42:48.911161 2745 catalog_manager.cc:1347] Generated new cluster ID: 6e13939ca34a4fda81c637a2caaa9977
I20260810 06:42:48.911279 2745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:42:48.926010 2745 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:42:48.927646 2745 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:42:48.944666 2745 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c908675bb57d4ea8823ecb2a7984b588: Generated new TSK 0
I20260810 06:42:48.945657 2745 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:42:48.959862 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 2568
I20260810 06:42:48.960904 2698 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:49.056793 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 2568
W20260810 06:42:49.079453 2543 connection.cc:535] client connection to 127.0.0.1:38539 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38539): Connection reset by peer (error 104)
2026-08-10T06:42:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2159 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:42:49 dist-test-slave-4tdj krb5kdc[2762](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:42:49 dist-test-slave-4tdj krb5kdc[2762](info): set up 2 sockets
Aug 10 06:42:49 dist-test-slave-4tdj krb5kdc[2762](info): commencing operation
krb5kdc: starting...
W20260810 06:42:50.295315 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.005s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:42:50 dist-test-slave-4tdj krb5kdc[2762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:42:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:42:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:42:50.579084 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32899
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:50.912407 2791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:50.912688 2791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:50.912798 2791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:50.920001 2791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:42:50.920113 2791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:50.920168 2791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:50.920229 2791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:42:50.920274 2791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:42:50.929834 2791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:50.931012 2791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:50.932268 2791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:50.943532 2802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:50.943562 2806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:50.943626 2801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:50.944386 2791 server_base.cc:1034] running on GCE node
I20260810 06:42:50.945168 2791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:50.946554 2791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:50.947767 2791 hybrid_clock.cc:648] HybridClock initialized: now 1786344170947712 us; error 44 us; skew 500 ppm
I20260810 06:42:50.948110 2791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:50 dist-test-slave-4tdj krb5kdc[2762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344170, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:50.951133 2791 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:42:50.952539 2791 webserver.cc:460] Webserver started at http://127.0.0.1:46699/ using document root <none> and password file <none>
I20260810 06:42:50.953145 2791 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:50.953254 2791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:50.953510 2791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:50.956386 2791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "c8a790e0575c4d5897acc7a7ac89570b"
format_stamp: "Formatted at 2026-08-10 06:42:50 on dist-test-slave-4tdj"
server_key: "d669358f52cea660e803bcf00496a365"
server_key_iv: "2f85d4ea330cdd60799e210a55f7d8bc"
server_key_version: "encryptionkey@0"
I20260810 06:42:50.957103 2791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "c8a790e0575c4d5897acc7a7ac89570b"
format_stamp: "Formatted at 2026-08-10 06:42:50 on dist-test-slave-4tdj"
server_key: "d669358f52cea660e803bcf00496a365"
server_key_iv: "2f85d4ea330cdd60799e210a55f7d8bc"
server_key_version: "encryptionkey@0"
I20260810 06:42:50.962160 2791 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:42:50.965484 2817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:50.966935 2791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:42:50.967087 2791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "c8a790e0575c4d5897acc7a7ac89570b"
format_stamp: "Formatted at 2026-08-10 06:42:50 on dist-test-slave-4tdj"
server_key: "d669358f52cea660e803bcf00496a365"
server_key_iv: "2f85d4ea330cdd60799e210a55f7d8bc"
server_key_version: "encryptionkey@0"
I20260810 06:42:50.967267 2791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:50.987039 2791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:50.991081 2791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:50.991333 2791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:51.009732 2791 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32899
I20260810 06:42:51.009770 2910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32899 every 8 connection(s)
I20260810 06:42:51.011173 2791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:42:51.012935 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 2791
I20260810 06:42:51.013120 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:42:51.013597 31983 external_mini_cluster.cc:1428] Setting key fc431fa578e48c4ac22996da2ebc894f
I20260810 06:42:51.015323 2912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:51.029575 2912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b: Bootstrap starting.
I20260810 06:42:51.030634 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:51.015432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:51755 (local address 127.0.0.1:32899)
0810 06:42:51.016246 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:51.016261 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:51.016312 (+ 51us) server_negotiation.cc:407] Connection header received
0810 06:42:51.018339 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:51.018367 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:51.018895 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:51.019304 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:51.019926 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:51.021921 (+ 1995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:51.024539 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:51.025618 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:51.025795 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:51.027598 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:51.027607 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:51.027625 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:51.028291 (+ 666us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:51.028380 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:51.028416 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:51.028577 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:51.029198 (+ 621us) server_negotiation.cc:299] Negotiation successful
0810 06:42:51.029361 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":318,"threads_started":1}
I20260810 06:42:51.032965 2912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:51.034088 2912 log.cc:824] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:51.034483 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:42:51.036695 2912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b: No bootstrap required, opened a new log
I20260810 06:42:51.045193 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:51.038898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:43117 (local address 127.0.0.1:32899)
0810 06:42:51.039061 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:51.039066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:51.039404 (+ 338us) server_negotiation.cc:407] Connection header received
0810 06:42:51.039544 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:51.039549 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:51.039619 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:51.039725 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:51.040221 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:51.041362 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:51.042688 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:51.043349 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:51.043482 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:51.044464 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:51.044471 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:51.044476 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:51.044496 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:51.044547 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:51.044567 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:51.044678 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:51.044847 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 06:42:51.044896 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:42:51.045667 2912 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a790e0575c4d5897acc7a7ac89570b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32899 } }
I20260810 06:42:51.046020 2912 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:51.046132 2912 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8a790e0575c4d5897acc7a7ac89570b, State: Initialized, Role: FOLLOWER
I20260810 06:42:51.046788 2912 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a790e0575c4d5897acc7a7ac89570b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32899 } }
I20260810 06:42:51.047049 2912 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:42:51.047185 2912 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:42:51.047348 2912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:51.049474 2912 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a790e0575c4d5897acc7a7ac89570b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32899 } }
I20260810 06:42:51.049930 2912 leader_election.cc:302] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8a790e0575c4d5897acc7a7ac89570b; no voters:
I20260810 06:42:51.050611 2912 leader_election.cc:288] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:42:51.050740 2930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:51.052420 2912 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:42:51.052794 2930 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b [term 1 LEADER]: Becoming Leader. State: Replica: c8a790e0575c4d5897acc7a7ac89570b, State: Running, Role: LEADER
I20260810 06:42:51.053308 2930 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a790e0575c4d5897acc7a7ac89570b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32899 } }
I20260810 06:42:51.056212 2932 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8a790e0575c4d5897acc7a7ac89570b. Latest consensus state: current_term: 1 leader_uuid: "c8a790e0575c4d5897acc7a7ac89570b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a790e0575c4d5897acc7a7ac89570b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32899 } } }
I20260810 06:42:51.056246 2931 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8a790e0575c4d5897acc7a7ac89570b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a790e0575c4d5897acc7a7ac89570b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32899 } } }
I20260810 06:42:51.056608 2931 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:51.056605 2932 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:51.059626 2944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:42:51.063723 2944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:42:51.071784 2944 catalog_manager.cc:1347] Generated new cluster ID: 80f0059f33224e4db5cb71f844998e0e
I20260810 06:42:51.071882 2944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:42:51.111505 2944 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:42:51.112787 2944 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:42:51.121410 2944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8a790e0575c4d5897acc7a7ac89570b: Generated new TSK 0
I20260810 06:42:51.122195 2944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:42:51.133757 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 2791
W20260810 06:42:51.134933 2906 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.56:51755
W20260810 06:42:51.135617 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 2791: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:42:51.135735 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 2791
W20260810 06:42:51.159098 2760 connection.cc:535] client connection to 127.0.0.1:32899 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32899): Connection reset by peer (error 104)
2026-08-10T06:42:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2080 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:42:51 dist-test-slave-4tdj krb5kdc[2964](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:42:51 dist-test-slave-4tdj krb5kdc[2964](info): set up 2 sockets
Aug 10 06:42:51 dist-test-slave-4tdj krb5kdc[2964](info): commencing operation
krb5kdc: starting...
W20260810 06:42:52.370005 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.002s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:42:52 dist-test-slave-4tdj krb5kdc[2964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:42:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:42:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:42:52.651480 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43267
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:52.986835 2988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:52.987106 2988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:52.987244 2988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:52.994208 2988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:42:52.994328 2988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:52.994395 2988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:52.994455 2988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:42:52.994505 2988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:42:53.004307 2988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:53.005445 2988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:53.006723 2988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:53.017589 2996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:53.017546 2995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:53.017653 2998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:53.018456 2988 server_base.cc:1034] running on GCE node
I20260810 06:42:53.019225 2988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:53.020536 2988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:53.021747 2988 hybrid_clock.cc:648] HybridClock initialized: now 1786344173021676 us; error 54 us; skew 500 ppm
I20260810 06:42:53.022094 2988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:53 dist-test-slave-4tdj krb5kdc[2964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344173, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:53.025851 2988 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:42:53.027163 2988 webserver.cc:460] Webserver started at http://127.0.0.1:38323/ using document root <none> and password file <none>
I20260810 06:42:53.027752 2988 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:53.027854 2988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:53.028087 2988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:53.030862 2988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "7f0d1ef6ffc94674941cb37a951d9f70"
format_stamp: "Formatted at 2026-08-10 06:42:53 on dist-test-slave-4tdj"
server_key: "ad2442b1ecd92e42cfda5822aefd988f"
server_key_iv: "5bccaf7c104e6f0e101f507a50151c74"
server_key_version: "encryptionkey@0"
I20260810 06:42:53.031576 2988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "7f0d1ef6ffc94674941cb37a951d9f70"
format_stamp: "Formatted at 2026-08-10 06:42:53 on dist-test-slave-4tdj"
server_key: "ad2442b1ecd92e42cfda5822aefd988f"
server_key_iv: "5bccaf7c104e6f0e101f507a50151c74"
server_key_version: "encryptionkey@0"
I20260810 06:42:53.036338 2988 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:42:53.039582 3008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:53.040885 2988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:42:53.041023 2988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "7f0d1ef6ffc94674941cb37a951d9f70"
format_stamp: "Formatted at 2026-08-10 06:42:53 on dist-test-slave-4tdj"
server_key: "ad2442b1ecd92e42cfda5822aefd988f"
server_key_iv: "5bccaf7c104e6f0e101f507a50151c74"
server_key_version: "encryptionkey@0"
I20260810 06:42:53.041172 2988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:53.046450 2988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:53.050176 2988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:53.050451 2988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:53.066241 2988 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43267
I20260810 06:42:53.066311 3093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43267 every 8 connection(s)
I20260810 06:42:53.067469 2988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:42:53.067987 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 2988
I20260810 06:42:53.068148 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:42:53.068727 31983 external_mini_cluster.cc:1428] Setting key 870e689bc6f30468e5f0720884d7b2a5
I20260810 06:42:53.071393 3096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:53.077689 3102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:53.070612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:56037 (local address 127.0.0.1:43267)
0810 06:42:53.071480 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:53.071505 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:53.071855 (+ 350us) server_negotiation.cc:407] Connection header received
0810 06:42:53.074011 (+ 2156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:53.074037 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:53.074506 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:53.074933 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:53.075877 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:53.075886 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:53.075902 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:53.075920 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:53.076009 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:53.076034 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:53.076238 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:53.076363 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 06:42:53.076561 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":360,"threads_started":1}
I20260810 06:42:53.081271 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:42:53.083961 3096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70: Bootstrap starting.
I20260810 06:42:53.087054 3096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:53.087692 3102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:53.085369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:36631 (local address 127.0.0.1:43267)
0810 06:42:53.085545 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:53.085550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:53.085996 (+ 446us) server_negotiation.cc:407] Connection header received
0810 06:42:53.086117 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:53.086121 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:53.086173 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:53.086273 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:53.087084 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:53.087089 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:53.087093 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:53.087110 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:53.087153 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:53.087172 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:53.087294 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:53.087361 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 06:42:53.087422 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:42:53.088356 3096 log.cc:824] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:53.091051 3096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70: No bootstrap required, opened a new log
I20260810 06:42:53.097864 3096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f0d1ef6ffc94674941cb37a951d9f70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43267 } }
I20260810 06:42:53.098186 3096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:53.098284 3096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f0d1ef6ffc94674941cb37a951d9f70, State: Initialized, Role: FOLLOWER
I20260810 06:42:53.098778 3096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f0d1ef6ffc94674941cb37a951d9f70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43267 } }
I20260810 06:42:53.098974 3096 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:42:53.099108 3096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:42:53.099254 3096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:53.101434 3096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f0d1ef6ffc94674941cb37a951d9f70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43267 } }
I20260810 06:42:53.101882 3096 leader_election.cc:302] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f0d1ef6ffc94674941cb37a951d9f70; no voters:
I20260810 06:42:53.102414 3096 leader_election.cc:288] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:42:53.102519 3116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:53.103143 3116 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70 [term 1 LEADER]: Becoming Leader. State: Replica: 7f0d1ef6ffc94674941cb37a951d9f70, State: Running, Role: LEADER
I20260810 06:42:53.103628 3116 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f0d1ef6ffc94674941cb37a951d9f70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43267 } }
I20260810 06:42:53.104187 3096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:42:53.105906 3118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f0d1ef6ffc94674941cb37a951d9f70. Latest consensus state: current_term: 1 leader_uuid: "7f0d1ef6ffc94674941cb37a951d9f70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f0d1ef6ffc94674941cb37a951d9f70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43267 } } }
I20260810 06:42:53.106225 3118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:53.106957 3117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f0d1ef6ffc94674941cb37a951d9f70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f0d1ef6ffc94674941cb37a951d9f70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43267 } } }
I20260810 06:42:53.107229 3117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:53.107875 3132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:42:53.112979 3132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:42:53.121234 3132 catalog_manager.cc:1347] Generated new cluster ID: ab8bc89850fd4c209084b82657af67f3
I20260810 06:42:53.121347 3132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:42:53.167850 3132 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:42:53.168334 3132 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:42:53.175922 3132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f0d1ef6ffc94674941cb37a951d9f70: Generated new TSK 0
I20260810 06:42:53.176883 3132 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:42:53.190734 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 2988
W20260810 06:42:53.192209 3087 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.56:56037
W20260810 06:42:53.192871 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 2988: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:42:53.193028 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 2988
2026-08-10T06:42:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2059 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:42:53 dist-test-slave-4tdj krb5kdc[3168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:42:53 dist-test-slave-4tdj krb5kdc[3168](info): set up 2 sockets
Aug 10 06:42:53 dist-test-slave-4tdj krb5kdc[3168](info): commencing operation
krb5kdc: starting...
W20260810 06:42:54.429167 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:42:54 dist-test-slave-4tdj krb5kdc[3168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:42:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:42:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:42:54.706995 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40919
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:55.034439 3183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:55.034713 3183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:55.034858 3183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:55.041826 3183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:42:55.041937 3183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:55.042013 3183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:55.042101 3183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:42:55.042161 3183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:42:55.051978 3183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40919
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:55.053149 3183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:55.054417 3183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:55.065372 3200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:55.065425 3198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:55.065639 3197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:55.066308 3183 server_base.cc:1034] running on GCE node
I20260810 06:42:55.067072 3183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:55.068389 3183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:55.069594 3183 hybrid_clock.cc:648] HybridClock initialized: now 1786344175069528 us; error 55 us; skew 500 ppm
I20260810 06:42:55.069917 3183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:55 dist-test-slave-4tdj krb5kdc[3168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344175, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:55.072893 3183 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:42:55.074222 3183 webserver.cc:460] Webserver started at http://127.0.0.1:39955/ using document root <none> and password file <none>
I20260810 06:42:55.074860 3183 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:55.074965 3183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:55.075212 3183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:55.077963 3183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "8f1f410f0a3149eab4ea7fabb832e254"
format_stamp: "Formatted at 2026-08-10 06:42:55 on dist-test-slave-4tdj"
server_key: "7735a4da1b18867fa816fffcaf94e909"
server_key_iv: "e834ac082923edad7d6e3610758f8655"
server_key_version: "encryptionkey@0"
I20260810 06:42:55.078707 3183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "8f1f410f0a3149eab4ea7fabb832e254"
format_stamp: "Formatted at 2026-08-10 06:42:55 on dist-test-slave-4tdj"
server_key: "7735a4da1b18867fa816fffcaf94e909"
server_key_iv: "e834ac082923edad7d6e3610758f8655"
server_key_version: "encryptionkey@0"
I20260810 06:42:55.083418 3183 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 06:42:55.086651 3210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:55.087852 3183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:42:55.087996 3183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "8f1f410f0a3149eab4ea7fabb832e254"
format_stamp: "Formatted at 2026-08-10 06:42:55 on dist-test-slave-4tdj"
server_key: "7735a4da1b18867fa816fffcaf94e909"
server_key_iv: "e834ac082923edad7d6e3610758f8655"
server_key_version: "encryptionkey@0"
I20260810 06:42:55.088164 3183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:55.116175 3183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:55.119993 3183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:55.120236 3183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:55.136758 3183 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40919
I20260810 06:42:55.136791 3294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40919 every 8 connection(s)
I20260810 06:42:55.138089 3183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:42:55.141750 3298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:55.142360 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 3183
I20260810 06:42:55.142562 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:42:55.142974 31983 external_mini_cluster.cc:1428] Setting key 5d1f8ef03132ac55823cd5d685bec323
I20260810 06:42:55.154785 3298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254: Bootstrap starting.
Aug 10 06:42:55 dist-test-slave-4tdj krb5kdc[3168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:42:55.158579 3298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:55.159749 3298 log.cc:824] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:55.162806 3298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254: No bootstrap required, opened a new log
I20260810 06:42:55.166491 3302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:55.144643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:35409 (local address 127.0.0.1:40919)
0810 06:42:55.145404 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:55.145414 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:55.145454 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:42:55.147538 (+ 2084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:55.147561 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:55.147954 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:55.148279 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:55.150003 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:55.151377 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:55.153188 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:55.153858 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:55.153983 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:55.158894 (+ 4911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:55.158901 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:55.158914 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:55.158952 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:55.162109 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:55.163089 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:55.163095 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:55.163101 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:55.163277 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:55.163798 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:55.163802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:55.163804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:55.164322 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:55.164594 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:55.165092 (+ 498us) server_negotiation.cc:299] Negotiation successful
0810 06:42:55.165408 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":316,"threads_started":1}
I20260810 06:42:55.170384 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:42:55.171962 3298 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f1f410f0a3149eab4ea7fabb832e254" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40919 } }
I20260810 06:42:55.172367 3298 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:55.172458 3298 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f1f410f0a3149eab4ea7fabb832e254, State: Initialized, Role: FOLLOWER
I20260810 06:42:55.173070 3298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f1f410f0a3149eab4ea7fabb832e254" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40919 } }
I20260810 06:42:55.173357 3298 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:42:55.173512 3298 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:42:55.173712 3298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:55.176460 3298 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f1f410f0a3149eab4ea7fabb832e254" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40919 } }
I20260810 06:42:55.176944 3298 leader_election.cc:302] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f1f410f0a3149eab4ea7fabb832e254; no voters:
I20260810 06:42:55.177505 3298 leader_election.cc:288] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:42:55.177678 3318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:55.179694 3298 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:42:55.185041 3318 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254 [term 1 LEADER]: Becoming Leader. State: Replica: 8f1f410f0a3149eab4ea7fabb832e254, State: Running, Role: LEADER
I20260810 06:42:55.185662 3318 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f1f410f0a3149eab4ea7fabb832e254" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40919 } }
I20260810 06:42:55.188308 3322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f1f410f0a3149eab4ea7fabb832e254" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f1f410f0a3149eab4ea7fabb832e254" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40919 } } }
I20260810 06:42:55.188716 3322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:55.188979 3324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f1f410f0a3149eab4ea7fabb832e254. Latest consensus state: current_term: 1 leader_uuid: "8f1f410f0a3149eab4ea7fabb832e254" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f1f410f0a3149eab4ea7fabb832e254" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40919 } } }
I20260810 06:42:55.189271 3324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:55.192921 3302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:55.175515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:42965 (local address 127.0.0.1:40919)
0810 06:42:55.175686 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:55.175691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:55.176060 (+ 369us) server_negotiation.cc:407] Connection header received
0810 06:42:55.176211 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:55.176215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:55.176288 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:55.176391 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:55.177982 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:55.179298 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:55.181220 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:55.181971 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:55.182085 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:55.185464 (+ 3379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:55.185471 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:55.185477 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:42:55.185518 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:55.188672 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:55.189653 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:55.189658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:55.189661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:55.189807 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:42:55.190427 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:42:55.190432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:42:55.190436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:42:55.190896 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:55.191157 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:55.192336 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0810 06:42:55.192528 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:42:55.197397 3349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:42:55.202809 3349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:42:55.210549 3349 catalog_manager.cc:1347] Generated new cluster ID: 453d722de29e45b79bcba47aa3fb2847
I20260810 06:42:55.210676 3349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:42:55.224776 3349 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:42:55.226096 3349 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:42:55.243062 3349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f1f410f0a3149eab4ea7fabb832e254: Generated new TSK 0
I20260810 06:42:55.243920 3349 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:42:55.258240 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 3183
I20260810 06:42:55.259261 3285 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:42:55.349419 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 3183
W20260810 06:42:55.370658 3163 connection.cc:535] client connection to 127.0.0.1:40919 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40919): Connection reset by peer (error 104)
2026-08-10T06:42:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2155 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:42:55 dist-test-slave-4tdj krb5kdc[3368](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:42:55 dist-test-slave-4tdj krb5kdc[3368](info): set up 2 sockets
Aug 10 06:42:55 dist-test-slave-4tdj krb5kdc[3368](info): commencing operation
krb5kdc: starting...
W20260810 06:42:56.578138 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:42:56 dist-test-slave-4tdj krb5kdc[3368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:42:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:42:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:42:56.845360 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45153
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:57.168089 3391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:57.168354 3391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:57.168485 3391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:57.175420 3391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:42:57.175519 3391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:57.175591 3391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:57.175688 3391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:42:57.175743 3391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:42:57.185040 3391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45153
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:57.186257 3391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:57.187517 3391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:57.198570 3400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:57.198642 3404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:57.198619 3406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:57.199878 3391 server_base.cc:1034] running on GCE node
I20260810 06:42:57.200654 3391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:57.202018 3391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:57.203253 3391 hybrid_clock.cc:648] HybridClock initialized: now 1786344177203184 us; error 50 us; skew 500 ppm
I20260810 06:42:57.203612 3391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:57 dist-test-slave-4tdj krb5kdc[3368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344177, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:57.206717 3391 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:42:57.208081 3391 webserver.cc:460] Webserver started at http://127.0.0.1:44611/ using document root <none> and password file <none>
I20260810 06:42:57.208724 3391 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:57.208845 3391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:57.209173 3391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:57.212106 3391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "85eda60299254b6faff2b1142e090930"
format_stamp: "Formatted at 2026-08-10 06:42:57 on dist-test-slave-4tdj"
server_key: "c61063c9e8c92e1a212b8a88e57e2516"
server_key_iv: "6536f6408c4dafa3de9522d96cacc94c"
server_key_version: "encryptionkey@0"
I20260810 06:42:57.212819 3391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "85eda60299254b6faff2b1142e090930"
format_stamp: "Formatted at 2026-08-10 06:42:57 on dist-test-slave-4tdj"
server_key: "c61063c9e8c92e1a212b8a88e57e2516"
server_key_iv: "6536f6408c4dafa3de9522d96cacc94c"
server_key_version: "encryptionkey@0"
I20260810 06:42:57.217777 3391 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:42:57.221225 3421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:57.222534 3391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:42:57.222702 3391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "85eda60299254b6faff2b1142e090930"
format_stamp: "Formatted at 2026-08-10 06:42:57 on dist-test-slave-4tdj"
server_key: "c61063c9e8c92e1a212b8a88e57e2516"
server_key_iv: "6536f6408c4dafa3de9522d96cacc94c"
server_key_version: "encryptionkey@0"
I20260810 06:42:57.222890 3391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:57.239205 3391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:57.243199 3391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:57.243453 3391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:57.259894 3391 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45153
I20260810 06:42:57.259900 3497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45153 every 8 connection(s)
I20260810 06:42:57.261210 3391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:42:57.264885 3498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:57.270177 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 3391
I20260810 06:42:57.270371 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:42:57.270824 31983 external_mini_cluster.cc:1428] Setting key ec3a49e3c2e304300b01a0a2cf540f3c
I20260810 06:42:57.275417 3498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930: Bootstrap starting.
I20260810 06:42:57.278929 3498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:57.280052 3498 log.cc:824] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:57.283020 3498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930: No bootstrap required, opened a new log
I20260810 06:42:57.283627 3503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:57.272495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:48977 (local address 127.0.0.1:45153)
0810 06:42:57.273294 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:57.273305 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:57.273348 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:42:57.275017 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:57.275038 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:57.275420 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:57.275763 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:57.276327 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:57.277591 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:57.279398 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:57.280044 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:57.280197 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:57.281444 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:57.281451 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:57.281464 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:57.281957 (+ 493us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:57.282031 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:57.282063 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:57.282182 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:57.282469 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 06:42:57.282644 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":350,"threads_started":1}
I20260810 06:42:57.287467 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:42:57.290928 3498 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85eda60299254b6faff2b1142e090930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45153 } }
I20260810 06:42:57.291633 3498 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:57.291812 3498 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85eda60299254b6faff2b1142e090930, State: Initialized, Role: FOLLOWER
I20260810 06:42:57.292390 3498 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85eda60299254b6faff2b1142e090930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45153 } }
I20260810 06:42:57.292635 3498 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:42:57.292763 3498 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:42:57.292919 3498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:57.295308 3498 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85eda60299254b6faff2b1142e090930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45153 } }
I20260810 06:42:57.295781 3498 leader_election.cc:302] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85eda60299254b6faff2b1142e090930; no voters:
I20260810 06:42:57.296391 3498 leader_election.cc:288] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:42:57.296487 3515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:57.297937 3503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:57.291548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:48435 (local address 127.0.0.1:45153)
0810 06:42:57.291706 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:57.291712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:57.292084 (+ 372us) server_negotiation.cc:407] Connection header received
0810 06:42:57.292263 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:57.292266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:57.292323 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:57.292401 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:57.292965 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:57.293778 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:57.295160 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:42:57.295654 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:42:57.295759 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:42:57.296842 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:57.296849 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:57.296854 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:57.296878 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:57.296926 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:57.296949 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:57.297595 (+ 646us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:57.297649 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 06:42:57.297688 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":47}
I20260810 06:42:57.298743 3498 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:42:57.298738 3515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930 [term 1 LEADER]: Becoming Leader. State: Replica: 85eda60299254b6faff2b1142e090930, State: Running, Role: LEADER
I20260810 06:42:57.299371 3515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85eda60299254b6faff2b1142e090930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45153 } }
I20260810 06:42:57.302225 3517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85eda60299254b6faff2b1142e090930" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85eda60299254b6faff2b1142e090930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45153 } } }
I20260810 06:42:57.302325 3521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85eda60299254b6faff2b1142e090930. Latest consensus state: current_term: 1 leader_uuid: "85eda60299254b6faff2b1142e090930" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85eda60299254b6faff2b1142e090930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45153 } } }
I20260810 06:42:57.302680 3521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:57.302680 3517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:57.304152 3535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:42:57.308761 3535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:42:57.316828 3535 catalog_manager.cc:1347] Generated new cluster ID: f5615f1d822d4658a1cb8f898112c85f
I20260810 06:42:57.316929 3535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:42:57.344992 3535 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:42:57.346256 3535 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:42:57.354667 3535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85eda60299254b6faff2b1142e090930: Generated new TSK 0
I20260810 06:42:57.355427 3535 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:42:57.365636 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 3391
W20260810 06:42:57.366775 3492 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.56:48977
W20260810 06:42:57.367467 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 3391: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:42:57.367599 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 3391
W20260810 06:42:57.391011 3365 connection.cc:535] client connection to 127.0.0.1:45153 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45153): Connection reset by peer (error 104)
2026-08-10T06:42:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2016 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:42:57 dist-test-slave-4tdj krb5kdc[3555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:42:57 dist-test-slave-4tdj krb5kdc[3555](info): set up 2 sockets
Aug 10 06:42:57 dist-test-slave-4tdj krb5kdc[3555](info): commencing operation
krb5kdc: starting...
W20260810 06:42:58.605791 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.013s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:42:58 dist-test-slave-4tdj krb5kdc[3555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:42:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:42:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:42:58.882005 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42045
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:42:59.207170 3585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:42:59.207430 3585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:42:59.207568 3585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:42:59.214468 3585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:42:59.214582 3585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:42:59.214656 3585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:42:59.214738 3585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:42:59.214795 3585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:42:59.224319 3585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42045
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:42:59.225577 3585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:42:59.226897 3585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:42:59.238183 3594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:59.238250 3595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:42:59.238204 3597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:42:59.238963 3585 server_base.cc:1034] running on GCE node
I20260810 06:42:59.239656 3585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:42:59.240978 3585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:42:59.242182 3585 hybrid_clock.cc:648] HybridClock initialized: now 1786344179242137 us; error 32 us; skew 500 ppm
I20260810 06:42:59.242532 3585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:42:59 dist-test-slave-4tdj krb5kdc[3555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344179, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:42:59.250342 3585 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:42:59.251803 3585 webserver.cc:460] Webserver started at http://127.0.0.1:35167/ using document root <none> and password file <none>
I20260810 06:42:59.252460 3585 fs_manager.cc:359] Metadata directory not provided
I20260810 06:42:59.252580 3585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:42:59.252866 3585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:42:59.255928 3585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "f15121fc755f475db831d4913f442065"
format_stamp: "Formatted at 2026-08-10 06:42:59 on dist-test-slave-4tdj"
server_key: "5b48ac55b06e7b117f4b7c027e9359af"
server_key_iv: "7ebe5d81cdce403eb0d3001e189386b9"
server_key_version: "encryptionkey@0"
I20260810 06:42:59.256675 3585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "f15121fc755f475db831d4913f442065"
format_stamp: "Formatted at 2026-08-10 06:42:59 on dist-test-slave-4tdj"
server_key: "5b48ac55b06e7b117f4b7c027e9359af"
server_key_iv: "7ebe5d81cdce403eb0d3001e189386b9"
server_key_version: "encryptionkey@0"
I20260810 06:42:59.261770 3585 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:42:59.265195 3606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:42:59.266505 3585 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:42:59.266657 3585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "f15121fc755f475db831d4913f442065"
format_stamp: "Formatted at 2026-08-10 06:42:59 on dist-test-slave-4tdj"
server_key: "5b48ac55b06e7b117f4b7c027e9359af"
server_key_iv: "7ebe5d81cdce403eb0d3001e189386b9"
server_key_version: "encryptionkey@0"
I20260810 06:42:59.266832 3585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:42:59.272298 3585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:42:59.275954 3585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:42:59.276216 3585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:42:59.293217 3585 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42045
I20260810 06:42:59.293263 3700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42045 every 8 connection(s)
I20260810 06:42:59.294576 3585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:42:59.297017 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 3585
I20260810 06:42:59.297200 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:42:59.297748 31983 external_mini_cluster.cc:1428] Setting key 7162867f9a44513b5561562854b97385
I20260810 06:42:59.298554 3703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:42:59.305086 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:59.299494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:55491 (local address 127.0.0.1:42045)
0810 06:42:59.300246 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:59.300256 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:59.300296 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:42:59.301851 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:59.301871 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:59.302200 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:59.302612 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:59.303563 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:59.303570 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:59.303582 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:59.303599 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:59.303671 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:59.303689 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:59.303902 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:59.304049 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 06:42:59.304229 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":310,"threads_started":1}
I20260810 06:42:59.308360 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:42:59.310583 3703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065: Bootstrap starting.
I20260810 06:42:59.313829 3703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065: Neither blocks nor log segments found. Creating new log.
I20260810 06:42:59.314718 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:42:59.312400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:57835 (local address 127.0.0.1:42045)
0810 06:42:59.312541 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 06:42:59.312546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:42:59.312957 (+ 411us) server_negotiation.cc:407] Connection header received
0810 06:42:59.313115 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:42:59.313119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:42:59.313160 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:42:59.313234 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:42:59.314089 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:42:59.314094 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:42:59.314099 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:42:59.314121 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:42:59.314161 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:42:59.314177 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:42:59.314278 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:42:59.314414 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 06:42:59.314468 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:42:59.315023 3703 log.cc:824] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065: Log is configured to *not* fsync() on all Append() calls
I20260810 06:42:59.317603 3703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065: No bootstrap required, opened a new log
I20260810 06:42:59.324759 3703 raft_consensus.cc:348] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f15121fc755f475db831d4913f442065" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42045 } }
I20260810 06:42:59.325109 3703 raft_consensus.cc:374] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:42:59.325197 3703 raft_consensus.cc:729] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f15121fc755f475db831d4913f442065, State: Initialized, Role: FOLLOWER
I20260810 06:42:59.325838 3703 consensus_queue.cc:260] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f15121fc755f475db831d4913f442065" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42045 } }
I20260810 06:42:59.326129 3703 raft_consensus.cc:388] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:42:59.326274 3703 raft_consensus.cc:482] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:42:59.326447 3703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:42:59.328433 3703 raft_consensus.cc:504] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f15121fc755f475db831d4913f442065" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42045 } }
I20260810 06:42:59.328840 3703 leader_election.cc:302] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f15121fc755f475db831d4913f442065; no voters:
I20260810 06:42:59.329332 3703 leader_election.cc:288] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:42:59.329457 3726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:42:59.330042 3726 raft_consensus.cc:686] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065 [term 1 LEADER]: Becoming Leader. State: Replica: f15121fc755f475db831d4913f442065, State: Running, Role: LEADER
I20260810 06:42:59.330621 3726 consensus_queue.cc:237] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f15121fc755f475db831d4913f442065" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42045 } }
I20260810 06:42:59.331210 3703 sys_catalog.cc:564] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:42:59.333575 3727 sys_catalog.cc:455] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f15121fc755f475db831d4913f442065" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f15121fc755f475db831d4913f442065" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42045 } } }
I20260810 06:42:59.333437 3728 sys_catalog.cc:455] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f15121fc755f475db831d4913f442065. Latest consensus state: current_term: 1 leader_uuid: "f15121fc755f475db831d4913f442065" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f15121fc755f475db831d4913f442065" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42045 } } }
I20260810 06:42:59.333884 3727 sys_catalog.cc:458] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:59.333936 3728 sys_catalog.cc:458] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065 [sys.catalog]: This master's current role is: LEADER
I20260810 06:42:59.335922 3737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:42:59.341018 3737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:42:59.349733 3737 catalog_manager.cc:1347] Generated new cluster ID: 43adda7bc502458bb09924b54232efe5
I20260810 06:42:59.349844 3737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:42:59.366204 3737 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:42:59.366605 3737 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:42:59.376636 3737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f15121fc755f475db831d4913f442065: Generated new TSK 0
I20260810 06:42:59.377327 3737 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:42:59.381289 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 3585
W20260810 06:42:59.382579 3693 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.56:55491
W20260810 06:42:59.383275 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 3585: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:42:59.383404 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 3585
2026-08-10T06:42:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2016 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:42:59 dist-test-slave-4tdj krb5kdc[3757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:42:59 dist-test-slave-4tdj krb5kdc[3757](info): set up 2 sockets
Aug 10 06:42:59 dist-test-slave-4tdj krb5kdc[3757](info): commencing operation
krb5kdc: starting...
W20260810 06:43:00.621876 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:43:00 dist-test-slave-4tdj krb5kdc[3757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:43:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:43:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:43:00.892938 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:35171
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:35929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:35171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:01.227487 3777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:01.227773 3777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:01.227880 3777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:01.235006 3777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:01.235116 3777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:01.235168 3777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:01.235227 3777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:01.235270 3777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:01.245016 3777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:35929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:35171
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:35171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:01.246335 3777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:01.247694 3777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:01.259452 3785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:01.259375 3786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:01.259421 3788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:01.260129 3777 server_base.cc:1034] running on GCE node
I20260810 06:43:01.260919 3777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:01.262207 3777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:01.263404 3777 hybrid_clock.cc:648] HybridClock initialized: now 1786344181263361 us; error 29 us; skew 500 ppm
I20260810 06:43:01.263753 3777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:01 dist-test-slave-4tdj krb5kdc[3757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344181, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:01.266755 3777 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:43:01.268213 3777 webserver.cc:460] Webserver started at http://127.31.59.254:46647/ using document root <none> and password file <none>
I20260810 06:43:01.268847 3777 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:01.268952 3777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:01.269192 3777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:01.272154 3777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "eaf404d90a1b4f2f979b0cbf2bcdc7df"
format_stamp: "Formatted at 2026-08-10 06:43:01 on dist-test-slave-4tdj"
server_key: "cae4221da6df6707a99a762847848e82"
server_key_iv: "44f19c64fa8e711d75c16c9cb68ba64b"
server_key_version: "encryptionkey@0"
I20260810 06:43:01.272866 3777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "eaf404d90a1b4f2f979b0cbf2bcdc7df"
format_stamp: "Formatted at 2026-08-10 06:43:01 on dist-test-slave-4tdj"
server_key: "cae4221da6df6707a99a762847848e82"
server_key_iv: "44f19c64fa8e711d75c16c9cb68ba64b"
server_key_version: "encryptionkey@0"
I20260810 06:43:01.277894 3777 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:43:01.281441 3806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:01.282856 3777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:43:01.283006 3777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "eaf404d90a1b4f2f979b0cbf2bcdc7df"
format_stamp: "Formatted at 2026-08-10 06:43:01 on dist-test-slave-4tdj"
server_key: "cae4221da6df6707a99a762847848e82"
server_key_iv: "44f19c64fa8e711d75c16c9cb68ba64b"
server_key_version: "encryptionkey@0"
I20260810 06:43:01.283183 3777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:01.307524 3777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:01.311295 3777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:01.311554 3777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:01.328737 3777 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:35171
I20260810 06:43:01.328769 3886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:35171 every 8 connection(s)
I20260810 06:43:01.330006 3777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:43:01.332043 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 3777
I20260810 06:43:01.332242 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:43:01.332792 31983 external_mini_cluster.cc:1428] Setting key e0ce08378cf54d2d83b05c026daea4a8
I20260810 06:43:01.334256 3888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:43:01 dist-test-slave-4tdj krb5kdc[3757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:43:01.347530 3888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df: Bootstrap starting.
I20260810 06:43:01.350908 3888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:01.351902 3888 log.cc:824] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:01.355031 3888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df: No bootstrap required, opened a new log
I20260810 06:43:01.355232 3893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:01.334540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53575 (local address 127.31.59.254:35171)
0810 06:43:01.335295 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:01.335307 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:01.335346 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:43:01.336910 (+ 1564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:01.336931 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:01.337304 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:01.337635 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:01.339438 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:01.340794 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:01.342466 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:01.343104 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:01.343223 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:01.348046 (+ 4823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:01.348053 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:01.348066 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:01.348104 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:01.350998 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:01.351746 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:01.351752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:01.351759 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:01.351928 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:01.352471 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:01.352477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:01.352480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:01.353220 (+ 740us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:01.353460 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:01.353943 (+ 483us) server_negotiation.cc:299] Negotiation successful
0810 06:43:01.354234 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":329,"threads_started":1}
I20260810 06:43:01.358817 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:43:01.364063 3888 raft_consensus.cc:348] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaf404d90a1b4f2f979b0cbf2bcdc7df" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35171 } }
I20260810 06:43:01.364399 3888 raft_consensus.cc:374] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:01.364503 3888 raft_consensus.cc:729] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf404d90a1b4f2f979b0cbf2bcdc7df, State: Initialized, Role: FOLLOWER
I20260810 06:43:01.365114 3888 consensus_queue.cc:260] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaf404d90a1b4f2f979b0cbf2bcdc7df" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35171 } }
I20260810 06:43:01.365408 3888 raft_consensus.cc:388] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:43:01.365569 3888 raft_consensus.cc:482] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:43:01.365744 3888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:01.368602 3888 raft_consensus.cc:504] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaf404d90a1b4f2f979b0cbf2bcdc7df" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35171 } }
I20260810 06:43:01.369163 3888 leader_election.cc:302] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eaf404d90a1b4f2f979b0cbf2bcdc7df; no voters:
I20260810 06:43:01.369740 3888 leader_election.cc:288] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:43:01.369867 3906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:01.372025 3888 sys_catalog.cc:564] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:43:01.372946 3906 raft_consensus.cc:686] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df [term 1 LEADER]: Becoming Leader. State: Replica: eaf404d90a1b4f2f979b0cbf2bcdc7df, State: Running, Role: LEADER
I20260810 06:43:01.374045 3906 consensus_queue.cc:237] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaf404d90a1b4f2f979b0cbf2bcdc7df" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35171 } }
I20260810 06:43:01.378321 3909 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df [sys.catalog]: SysCatalogTable state changed. Reason: New leader eaf404d90a1b4f2f979b0cbf2bcdc7df. Latest consensus state: current_term: 1 leader_uuid: "eaf404d90a1b4f2f979b0cbf2bcdc7df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaf404d90a1b4f2f979b0cbf2bcdc7df" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35171 } } }
I20260810 06:43:01.378749 3909 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:01.379287 3908 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eaf404d90a1b4f2f979b0cbf2bcdc7df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaf404d90a1b4f2f979b0cbf2bcdc7df" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35171 } } }
I20260810 06:43:01.379549 3908 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:01.381451 3893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:01.363826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43887 (local address 127.31.59.254:35171)
0810 06:43:01.363994 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:01.363999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:01.364452 (+ 453us) server_negotiation.cc:407] Connection header received
0810 06:43:01.364639 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:01.364645 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:01.364704 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:01.364790 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:01.366663 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:01.367449 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:01.369105 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:01.369659 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:01.369800 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:01.372780 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:01.372787 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:01.372793 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:01.372835 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:01.376971 (+ 4136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:01.378073 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:01.378094 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:01.378098 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:01.378282 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:01.378955 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:01.378960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:01.378963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:01.379386 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:01.380668 (+ 1282us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:01.380848 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 06:43:01.381035 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:43:01.388350 3922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:43:01.393560 3922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:43:01.402204 3922 catalog_manager.cc:1347] Generated new cluster ID: 7d9ce5fbb6d24301b77f2acc63d79f42
I20260810 06:43:01.402330 3922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:43:01.419093 3922 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:43:01.420382 3922 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:43:01.431867 3922 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eaf404d90a1b4f2f979b0cbf2bcdc7df: Generated new TSK 0
I20260810 06:43:01.432667 3922 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:43:01.446561 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 3777
I20260810 06:43:01.447703 3882 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:01.538362 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 3777
W20260810 06:43:01.563412 3753 connection.cc:535] client connection to 127.31.59.254:35171 recv error: Network error: failed to read from TLS socket (remote: 127.31.59.254:35171): Connection reset by peer (error 104)
2026-08-10T06:43:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2155 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:43:01 dist-test-slave-4tdj krb5kdc[3951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:43:01 dist-test-slave-4tdj krb5kdc[3951](info): set up 2 sockets
Aug 10 06:43:01 dist-test-slave-4tdj krb5kdc[3951](info): commencing operation
krb5kdc: starting...
W20260810 06:43:02.794339 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:43:02 dist-test-slave-4tdj krb5kdc[3951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:43:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:43:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:43:03.089010 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:45855
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:41227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:45855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:03.420246 3978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:03.420542 3978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:03.420691 3978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:03.427815 3978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:03.427939 3978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:03.428016 3978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:03.428107 3978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:03.428167 3978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:03.437932 3978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:41227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:45855
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:45855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:03.439214 3978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:03.440491 3978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:03.451454 3985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:03.451527 3991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:03.451486 3989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:03.452569 3978 server_base.cc:1034] running on GCE node
I20260810 06:43:03.453260 3978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:03.454666 3978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:03.455861 3978 hybrid_clock.cc:648] HybridClock initialized: now 1786344183455819 us; error 30 us; skew 500 ppm
I20260810 06:43:03.456197 3978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:03 dist-test-slave-4tdj krb5kdc[3951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344183, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:03.459156 3978 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:43:03.460549 3978 webserver.cc:460] Webserver started at http://127.31.59.254:45963/ using document root <none> and password file <none>
I20260810 06:43:03.461189 3978 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:03.461306 3978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:03.461586 3978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:03.464466 3978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "0171d56451c8489cbae9852208bf9f8a"
format_stamp: "Formatted at 2026-08-10 06:43:03 on dist-test-slave-4tdj"
server_key: "73875a37368305334368d3b30a88aa1a"
server_key_iv: "c603db27012e7db53be977f26cd91c31"
server_key_version: "encryptionkey@0"
I20260810 06:43:03.465286 3978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "0171d56451c8489cbae9852208bf9f8a"
format_stamp: "Formatted at 2026-08-10 06:43:03 on dist-test-slave-4tdj"
server_key: "73875a37368305334368d3b30a88aa1a"
server_key_iv: "c603db27012e7db53be977f26cd91c31"
server_key_version: "encryptionkey@0"
I20260810 06:43:03.470252 3978 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 06:43:03.473569 4001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:03.475090 3978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:43:03.475260 3978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "0171d56451c8489cbae9852208bf9f8a"
format_stamp: "Formatted at 2026-08-10 06:43:03 on dist-test-slave-4tdj"
server_key: "73875a37368305334368d3b30a88aa1a"
server_key_iv: "c603db27012e7db53be977f26cd91c31"
server_key_version: "encryptionkey@0"
I20260810 06:43:03.475453 3978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:03.506409 3978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:03.511950 3978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:03.512243 3978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:03.529280 3978 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:45855
I20260810 06:43:03.529304 4079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:45855 every 8 connection(s)
I20260810 06:43:03.530716 3978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:43:03.534544 4081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:03.536849 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 3978
I20260810 06:43:03.537034 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:43:03.537549 31983 external_mini_cluster.cc:1428] Setting key 59ad701d1ca92f196942f99920a28030
I20260810 06:43:03.545909 4081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a: Bootstrap starting.
I20260810 06:43:03.548981 4081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:03.550091 4081 log.cc:824] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:03.551065 4091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:03.539461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48549 (local address 127.31.59.254:45855)
0810 06:43:03.540315 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:03.540328 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:03.540377 (+ 49us) server_negotiation.cc:407] Connection header received
0810 06:43:03.542125 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:03.542147 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:03.542597 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:03.542986 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:03.543601 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:03.545016 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:03.546447 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:03.547140 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:03.547348 (+ 208us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:03.548617 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:03.548625 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:03.548638 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:43:03.549183 (+ 545us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:03.549264 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:43:03.549291 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:03.549445 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:03.549750 (+ 305us) server_negotiation.cc:299] Negotiation successful
0810 06:43:03.549902 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":383,"threads_started":1}
I20260810 06:43:03.553138 4081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a: No bootstrap required, opened a new log
I20260810 06:43:03.554701 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:43:03.561143 4081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0171d56451c8489cbae9852208bf9f8a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 45855 } }
I20260810 06:43:03.561615 4081 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:03.561722 4081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0171d56451c8489cbae9852208bf9f8a, State: Initialized, Role: FOLLOWER
I20260810 06:43:03.562378 4081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0171d56451c8489cbae9852208bf9f8a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 45855 } }
I20260810 06:43:03.562661 4081 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:43:03.562799 4081 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:43:03.563009 4081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:03.565323 4091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:03.558798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50519 (local address 127.31.59.254:45855)
0810 06:43:03.559002 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:03.559008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:03.559387 (+ 379us) server_negotiation.cc:407] Connection header received
0810 06:43:03.559617 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:03.559623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:03.559687 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:03.559777 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:03.560338 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:03.561072 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:03.562855 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:03.563354 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:03.563449 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:03.564586 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:03.564594 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:03.564598 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:43:03.564615 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:03.564669 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:43:03.564687 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:03.564770 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:03.564955 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 06:43:03.565013 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:43:03.565945 4081 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0171d56451c8489cbae9852208bf9f8a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 45855 } }
I20260810 06:43:03.566507 4081 leader_election.cc:302] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0171d56451c8489cbae9852208bf9f8a; no voters:
I20260810 06:43:03.567088 4081 leader_election.cc:288] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:43:03.567178 4103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:03.567770 4103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a [term 1 LEADER]: Becoming Leader. State: Replica: 0171d56451c8489cbae9852208bf9f8a, State: Running, Role: LEADER
I20260810 06:43:03.568385 4103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0171d56451c8489cbae9852208bf9f8a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 45855 } }
I20260810 06:43:03.569360 4081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:43:03.571171 4105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0171d56451c8489cbae9852208bf9f8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0171d56451c8489cbae9852208bf9f8a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 45855 } } }
I20260810 06:43:03.571523 4105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:03.572324 4106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0171d56451c8489cbae9852208bf9f8a. Latest consensus state: current_term: 1 leader_uuid: "0171d56451c8489cbae9852208bf9f8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0171d56451c8489cbae9852208bf9f8a" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 45855 } } }
I20260810 06:43:03.572710 4106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:03.573077 4113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:43:03.578465 4113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:43:03.586865 4113 catalog_manager.cc:1347] Generated new cluster ID: 258e9dc6023a4b0ab845a0417ca9f84b
I20260810 06:43:03.586973 4113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:43:03.600844 4113 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:43:03.602556 4113 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:43:03.611122 4113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0171d56451c8489cbae9852208bf9f8a: Generated new TSK 0
I20260810 06:43:03.611832 4113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:43:03.618117 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 3978
W20260810 06:43:03.619172 4072 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48549
W20260810 06:43:03.619910 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 3978: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:43:03.620059 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 3978
W20260810 06:43:03.648893 3948 connection.cc:535] client connection to 127.31.59.254:45855 recv error: Network error: failed to read from TLS socket (remote: 127.31.59.254:45855): Connection reset by peer (error 104)
2026-08-10T06:43:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2085 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:43:03 dist-test-slave-4tdj krb5kdc[4135](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:43:03 dist-test-slave-4tdj krb5kdc[4135](info): set up 2 sockets
Aug 10 06:43:03 dist-test-slave-4tdj krb5kdc[4135](info): commencing operation
krb5kdc: starting...
W20260810 06:43:04.860100 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.004s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:43:05 dist-test-slave-4tdj krb5kdc[4135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:43:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:43:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:43:05.121345 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:38019
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:38019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:05.443630 4160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:05.443890 4160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:05.443985 4160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:05.450799 4160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:05.450891 4160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:05.450942 4160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:05.451000 4160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:05.451043 4160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:05.460151 4160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:38019
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:38019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:05.461242 4160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:05.462452 4160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:05.473541 4170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:05.473541 4174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:05.473693 4172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:05.481220 4160 server_base.cc:1034] running on GCE node
I20260810 06:43:05.481985 4160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:05.483362 4160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:05.484561 4160 hybrid_clock.cc:648] HybridClock initialized: now 1786344185484517 us; error 30 us; skew 500 ppm
I20260810 06:43:05.484869 4160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:05 dist-test-slave-4tdj krb5kdc[4135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344185, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:05.490895 4160 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:43:05.492379 4160 webserver.cc:460] Webserver started at http://127.31.59.254:37299/ using document root <none> and password file <none>
I20260810 06:43:05.493069 4160 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:05.493177 4160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:05.493465 4160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:05.496371 4160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "d714fd70b9444dcd9aadeef54a46fae1"
format_stamp: "Formatted at 2026-08-10 06:43:05 on dist-test-slave-4tdj"
server_key: "13f5e488d52e5b6627172b06d52180ba"
server_key_iv: "7fc3a46c36318138a2ae105fcb898276"
server_key_version: "encryptionkey@0"
I20260810 06:43:05.497119 4160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "d714fd70b9444dcd9aadeef54a46fae1"
format_stamp: "Formatted at 2026-08-10 06:43:05 on dist-test-slave-4tdj"
server_key: "13f5e488d52e5b6627172b06d52180ba"
server_key_iv: "7fc3a46c36318138a2ae105fcb898276"
server_key_version: "encryptionkey@0"
I20260810 06:43:05.502066 4160 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260810 06:43:05.505352 4186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:05.506645 4160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:43:05.506803 4160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "d714fd70b9444dcd9aadeef54a46fae1"
format_stamp: "Formatted at 2026-08-10 06:43:05 on dist-test-slave-4tdj"
server_key: "13f5e488d52e5b6627172b06d52180ba"
server_key_iv: "7fc3a46c36318138a2ae105fcb898276"
server_key_version: "encryptionkey@0"
I20260810 06:43:05.506978 4160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:05.512511 4160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:05.516376 4160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:05.516647 4160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:05.534314 4160 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:38019
I20260810 06:43:05.534376 4276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:38019 every 8 connection(s)
I20260810 06:43:05.535653 4160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:43:05.539270 4277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:05.544917 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 4160
I20260810 06:43:05.545109 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:43:05.545656 31983 external_mini_cluster.cc:1428] Setting key 39dfcea2ff04714c0d3d012cff0baa90
I20260810 06:43:05.549926 4277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1: Bootstrap starting.
I20260810 06:43:05.552911 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:05.547377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40749 (local address 127.31.59.254:38019)
0810 06:43:05.548229 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:05.548240 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:05.548278 (+ 38us) server_negotiation.cc:407] Connection header received
0810 06:43:05.549847 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:05.549868 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:05.550199 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:05.550547 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:05.551493 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:05.551502 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:05.551515 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:43:05.551527 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:05.551615 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:43:05.551636 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:05.551797 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:05.551894 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 06:43:05.552058 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":407,"threads_started":1}
I20260810 06:43:05.553357 4277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:05.554401 4277 log.cc:824] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:05.556314 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:43:05.556998 4277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1: No bootstrap required, opened a new log
I20260810 06:43:05.562674 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:05.560293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43211 (local address 127.31.59.254:38019)
0810 06:43:05.560449 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:05.560454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:05.560911 (+ 457us) server_negotiation.cc:407] Connection header received
0810 06:43:05.561033 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:05.561038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:05.561099 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:05.561203 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:05.561977 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:05.561982 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:05.561987 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:43:05.562004 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:05.562058 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:43:05.562081 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:05.562197 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:05.562324 (+ 127us) server_negotiation.cc:299] Negotiation successful
0810 06:43:05.562378 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:43:05.564859 4277 raft_consensus.cc:348] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d714fd70b9444dcd9aadeef54a46fae1" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38019 } }
I20260810 06:43:05.565174 4277 raft_consensus.cc:374] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:05.565289 4277 raft_consensus.cc:729] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d714fd70b9444dcd9aadeef54a46fae1, State: Initialized, Role: FOLLOWER
I20260810 06:43:05.565886 4277 consensus_queue.cc:260] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d714fd70b9444dcd9aadeef54a46fae1" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38019 } }
I20260810 06:43:05.566092 4277 raft_consensus.cc:388] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:43:05.566192 4277 raft_consensus.cc:482] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:43:05.566383 4277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:05.568421 4277 raft_consensus.cc:504] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d714fd70b9444dcd9aadeef54a46fae1" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38019 } }
I20260810 06:43:05.568856 4277 leader_election.cc:302] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d714fd70b9444dcd9aadeef54a46fae1; no voters:
I20260810 06:43:05.569461 4277 leader_election.cc:288] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:43:05.569547 4293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:05.570363 4293 raft_consensus.cc:686] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1 [term 1 LEADER]: Becoming Leader. State: Replica: d714fd70b9444dcd9aadeef54a46fae1, State: Running, Role: LEADER
I20260810 06:43:05.570873 4293 consensus_queue.cc:237] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d714fd70b9444dcd9aadeef54a46fae1" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38019 } }
I20260810 06:43:05.571424 4277 sys_catalog.cc:564] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:43:05.573520 4295 sys_catalog.cc:455] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d714fd70b9444dcd9aadeef54a46fae1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d714fd70b9444dcd9aadeef54a46fae1" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38019 } } }
I20260810 06:43:05.573938 4295 sys_catalog.cc:458] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:05.574580 4296 sys_catalog.cc:455] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d714fd70b9444dcd9aadeef54a46fae1. Latest consensus state: current_term: 1 leader_uuid: "d714fd70b9444dcd9aadeef54a46fae1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d714fd70b9444dcd9aadeef54a46fae1" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 38019 } } }
I20260810 06:43:05.574916 4296 sys_catalog.cc:458] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:05.576865 4306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:43:05.581779 4306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:43:05.589891 4306 catalog_manager.cc:1347] Generated new cluster ID: bf2e006a32464813bc68a9e2ca113839
I20260810 06:43:05.589996 4306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:43:05.631973 4306 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:43:05.632378 4306 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:43:05.643487 4306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d714fd70b9444dcd9aadeef54a46fae1: Generated new TSK 0
I20260810 06:43:05.644210 4306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:43:05.652727 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 4160
W20260810 06:43:05.654325 4262 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40749
W20260810 06:43:05.654991 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 4160: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:43:05.655128 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 4160
2026-08-10T06:43:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2032 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:43:05 dist-test-slave-4tdj krb5kdc[4325](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:43:05 dist-test-slave-4tdj krb5kdc[4325](info): set up 2 sockets
Aug 10 06:43:05 dist-test-slave-4tdj krb5kdc[4325](info): commencing operation
krb5kdc: starting...
W20260810 06:43:06.900928 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:43:07 dist-test-slave-4tdj krb5kdc[4325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:43:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:43:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:43:07.195262 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:46587
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:38495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:46587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:07.525456 4345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:07.525727 4345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:07.525825 4345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:07.533864 4345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:07.533982 4345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:07.534039 4345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:07.534101 4345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:07.534142 4345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:07.544307 4345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:38495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:46587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:46587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:07.545568 4345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:07.546953 4345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:07.558529 4356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:07.558660 4352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:07.558655 4354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:07.559607 4345 server_base.cc:1034] running on GCE node
I20260810 06:43:07.560382 4345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:07.561729 4345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:07.562935 4345 hybrid_clock.cc:648] HybridClock initialized: now 1786344187562883 us; error 41 us; skew 500 ppm
I20260810 06:43:07.563328 4345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:07 dist-test-slave-4tdj krb5kdc[4325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344187, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:07.566524 4345 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:43:07.567978 4345 webserver.cc:460] Webserver started at http://127.31.59.254:44095/ using document root <none> and password file <none>
I20260810 06:43:07.568642 4345 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:07.568773 4345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:07.569083 4345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:07.572242 4345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "95c1763d09ea45c793d9ad2f86efee48"
format_stamp: "Formatted at 2026-08-10 06:43:07 on dist-test-slave-4tdj"
server_key: "40129973b52222e9795c7a2b69a316b6"
server_key_iv: "14a920182cef8836aada78ea3f48c4da"
server_key_version: "encryptionkey@0"
I20260810 06:43:07.573048 4345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "95c1763d09ea45c793d9ad2f86efee48"
format_stamp: "Formatted at 2026-08-10 06:43:07 on dist-test-slave-4tdj"
server_key: "40129973b52222e9795c7a2b69a316b6"
server_key_iv: "14a920182cef8836aada78ea3f48c4da"
server_key_version: "encryptionkey@0"
I20260810 06:43:07.578238 4345 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 06:43:07.581676 4369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:07.582985 4345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:43:07.583141 4345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "95c1763d09ea45c793d9ad2f86efee48"
format_stamp: "Formatted at 2026-08-10 06:43:07 on dist-test-slave-4tdj"
server_key: "40129973b52222e9795c7a2b69a316b6"
server_key_iv: "14a920182cef8836aada78ea3f48c4da"
server_key_version: "encryptionkey@0"
I20260810 06:43:07.583317 4345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:07.595727 4345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:07.599637 4345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:07.599881 4345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:07.617049 4345 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:46587
I20260810 06:43:07.617069 4454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:46587 every 8 connection(s)
I20260810 06:43:07.618331 4345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:43:07.621814 4455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:07.623070 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 4345
I20260810 06:43:07.623245 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:43:07.623668 31983 external_mini_cluster.cc:1428] Setting key 6a38b3599f0808c35376500143893c9c
I20260810 06:43:07.635141 4455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48: Bootstrap starting.
Aug 10 06:43:07 dist-test-slave-4tdj krb5kdc[4325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:43:07.638558 4455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:07.639599 4455 log.cc:824] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:07.642612 4455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48: No bootstrap required, opened a new log
I20260810 06:43:07.645998 4460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:07.625384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58437 (local address 127.31.59.254:46587)
0810 06:43:07.626209 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:07.626220 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:07.626264 (+ 44us) server_negotiation.cc:407] Connection header received
0810 06:43:07.627844 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:07.627865 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:07.628248 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:07.628548 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:07.630284 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:07.631595 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:07.633338 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:07.633948 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:07.634081 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:07.638444 (+ 4363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:07.638455 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:07.638468 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:07.638506 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:07.641774 (+ 3268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:07.642424 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:07.642431 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:07.642439 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:07.642649 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:07.643024 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:07.643029 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:07.643043 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:07.643694 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:07.643978 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:07.644297 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 06:43:07.644720 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":360,"threads_started":1}
I20260810 06:43:07.649353 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:43:07.651018 4455 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c1763d09ea45c793d9ad2f86efee48" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 46587 } }
I20260810 06:43:07.651331 4455 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:07.651424 4455 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95c1763d09ea45c793d9ad2f86efee48, State: Initialized, Role: FOLLOWER
I20260810 06:43:07.651986 4455 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c1763d09ea45c793d9ad2f86efee48" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 46587 } }
I20260810 06:43:07.652208 4455 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:43:07.652343 4455 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:43:07.652484 4455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:07.654976 4455 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c1763d09ea45c793d9ad2f86efee48" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 46587 } }
I20260810 06:43:07.655506 4455 leader_election.cc:302] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95c1763d09ea45c793d9ad2f86efee48; no voters:
I20260810 06:43:07.656157 4455 leader_election.cc:288] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:43:07.656247 4473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:07.657951 4455 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:43:07.663293 4473 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48 [term 1 LEADER]: Becoming Leader. State: Replica: 95c1763d09ea45c793d9ad2f86efee48, State: Running, Role: LEADER
I20260810 06:43:07.663883 4473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c1763d09ea45c793d9ad2f86efee48" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 46587 } }
I20260810 06:43:07.666538 4474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95c1763d09ea45c793d9ad2f86efee48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c1763d09ea45c793d9ad2f86efee48" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 46587 } } }
I20260810 06:43:07.666977 4474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:07.671979 4460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:07.653400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34325 (local address 127.31.59.254:46587)
0810 06:43:07.653592 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:07.653599 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:07.654836 (+ 1237us) server_negotiation.cc:407] Connection header received
0810 06:43:07.655036 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:07.655040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:07.655115 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:07.655219 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:07.657093 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:07.658254 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:07.660018 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:07.660781 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:07.660921 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:07.664115 (+ 3194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:07.664123 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:07.664128 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:07.664170 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:07.668011 (+ 3841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:07.668882 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:07.668888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:07.668891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:07.669046 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:07.669522 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:07.669527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:07.669530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:07.669947 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:07.671222 (+ 1275us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:07.671374 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 06:43:07.671599 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:43:07.672057 4475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95c1763d09ea45c793d9ad2f86efee48. Latest consensus state: current_term: 1 leader_uuid: "95c1763d09ea45c793d9ad2f86efee48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c1763d09ea45c793d9ad2f86efee48" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 46587 } } }
I20260810 06:43:07.673012 4475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48 [sys.catalog]: This master's current role is: LEADER
W20260810 06:43:07.681588 4503 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:43:07.681754 4503 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:43:07.681875 4498 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:43:07.683897 4498 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:43:07.691742 4498 catalog_manager.cc:1347] Generated new cluster ID: 2b00dcadb74c416fb3dd2a134bc7556d
I20260810 06:43:07.691851 4498 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:43:07.737850 4498 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:43:07.739181 4498 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:43:07.757189 4498 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95c1763d09ea45c793d9ad2f86efee48: Generated new TSK 0
I20260810 06:43:07.758014 4498 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:43:07.763499 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 4345
I20260810 06:43:07.764700 4450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:07.857743 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 4345
2026-08-10T06:43:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2205 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:43:07 dist-test-slave-4tdj krb5kdc[4512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:43:07 dist-test-slave-4tdj krb5kdc[4512](info): set up 2 sockets
Aug 10 06:43:07 dist-test-slave-4tdj krb5kdc[4512](info): commencing operation
krb5kdc: starting...
W20260810 06:43:09.118389 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.004s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:43:09 dist-test-slave-4tdj krb5kdc[4512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:43:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:43:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:43:09.388975 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:35629
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:42155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:35629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:09.720019 4539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:09.720295 4539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:09.720415 4539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:09.727469 4539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:09.727569 4539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:09.727635 4539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:09.727718 4539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:09.727773 4539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:09.737330 4539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:42155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:35629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:35629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:09.738592 4539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:09.739887 4539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:09.750900 4547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:09.750941 4549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:09.750962 4554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:09.751672 4539 server_base.cc:1034] running on GCE node
I20260810 06:43:09.752329 4539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:09.753548 4539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:09.754733 4539 hybrid_clock.cc:648] HybridClock initialized: now 1786344189754697 us; error 28 us; skew 500 ppm
I20260810 06:43:09.755048 4539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:09 dist-test-slave-4tdj krb5kdc[4512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344189, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:09.757969 4539 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:43:09.759344 4539 webserver.cc:460] Webserver started at http://127.31.59.254:39369/ using document root <none> and password file <none>
I20260810 06:43:09.759963 4539 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:09.760061 4539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:09.760303 4539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:09.763141 4539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "4b60ebc02f4343cfa144f0b9a3f8f553"
format_stamp: "Formatted at 2026-08-10 06:43:09 on dist-test-slave-4tdj"
server_key: "9c143eb9a8f4d774ea97c7ee383c1b41"
server_key_iv: "19f5675bc0ebf998f5f0c4de952d4959"
server_key_version: "encryptionkey@0"
I20260810 06:43:09.763829 4539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "4b60ebc02f4343cfa144f0b9a3f8f553"
format_stamp: "Formatted at 2026-08-10 06:43:09 on dist-test-slave-4tdj"
server_key: "9c143eb9a8f4d774ea97c7ee383c1b41"
server_key_iv: "19f5675bc0ebf998f5f0c4de952d4959"
server_key_version: "encryptionkey@0"
I20260810 06:43:09.768693 4539 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:43:09.772006 4569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:09.773239 4539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:43:09.773393 4539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "4b60ebc02f4343cfa144f0b9a3f8f553"
format_stamp: "Formatted at 2026-08-10 06:43:09 on dist-test-slave-4tdj"
server_key: "9c143eb9a8f4d774ea97c7ee383c1b41"
server_key_iv: "19f5675bc0ebf998f5f0c4de952d4959"
server_key_version: "encryptionkey@0"
I20260810 06:43:09.773574 4539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:09.798784 4539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:09.803323 4539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:09.803754 4539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:09.831969 4539 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:35629
I20260810 06:43:09.832016 4645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:35629 every 8 connection(s)
I20260810 06:43:09.834154 4539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:43:09.840525 4647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:09.844655 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 4539
I20260810 06:43:09.844942 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:43:09.845934 31983 external_mini_cluster.cc:1428] Setting key b63e149382defd5ec0bdedc41216316b
I20260810 06:43:09.861073 4647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553: Bootstrap starting.
I20260810 06:43:09.867154 4647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:09.868265 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:09.849229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55745 (local address 127.31.59.254:35629)
0810 06:43:09.850548 (+ 1319us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:09.850576 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:09.850654 (+ 78us) server_negotiation.cc:407] Connection header received
0810 06:43:09.853352 (+ 2698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:09.853399 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:09.854079 (+ 680us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:09.854680 (+ 601us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:09.855648 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:09.857764 (+ 2116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:09.860779 (+ 3015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:09.862020 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:09.862241 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:09.864373 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:09.864390 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:09.864413 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:43:09.865295 (+ 882us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:09.865408 (+ 113us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:43:09.865454 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:09.865610 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:09.865984 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 06:43:09.866459 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":928,"thread_start_us":580,"threads_started":1}
I20260810 06:43:09.868986 4647 log.cc:824] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:09.874435 4647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553: No bootstrap required, opened a new log
I20260810 06:43:09.874708 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:43:09.889030 4647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b60ebc02f4343cfa144f0b9a3f8f553" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35629 } }
I20260810 06:43:09.889874 4647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:09.890062 4647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b60ebc02f4343cfa144f0b9a3f8f553, State: Initialized, Role: FOLLOWER
I20260810 06:43:09.891185 4647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b60ebc02f4343cfa144f0b9a3f8f553" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35629 } }
I20260810 06:43:09.891695 4647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:43:09.891945 4647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:43:09.892364 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:09.881619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36995 (local address 127.31.59.254:35629)
0810 06:43:09.881929 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:09.881947 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:09.882583 (+ 636us) server_negotiation.cc:407] Connection header received
0810 06:43:09.882855 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:09.882861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:09.882985 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:09.883186 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:09.883913 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:09.885781 (+ 1868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:09.887903 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:09.889065 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:09.889329 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:09.890950 (+ 1621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:09.890967 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:09.890972 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:43:09.891005 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:09.891103 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:43:09.891150 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:09.891340 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:09.891530 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 06:43:09.891704 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 06:43:09.892329 4647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:09.896407 4647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b60ebc02f4343cfa144f0b9a3f8f553" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35629 } }
I20260810 06:43:09.897193 4647 leader_election.cc:302] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b60ebc02f4343cfa144f0b9a3f8f553; no voters:
I20260810 06:43:09.897935 4647 leader_election.cc:288] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:43:09.898182 4663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:09.900238 4663 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553 [term 1 LEADER]: Becoming Leader. State: Replica: 4b60ebc02f4343cfa144f0b9a3f8f553, State: Running, Role: LEADER
I20260810 06:43:09.901098 4663 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b60ebc02f4343cfa144f0b9a3f8f553" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35629 } }
I20260810 06:43:09.901975 4647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:43:09.907812 4665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b60ebc02f4343cfa144f0b9a3f8f553. Latest consensus state: current_term: 1 leader_uuid: "4b60ebc02f4343cfa144f0b9a3f8f553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b60ebc02f4343cfa144f0b9a3f8f553" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35629 } } }
I20260810 06:43:09.907846 4664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b60ebc02f4343cfa144f0b9a3f8f553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b60ebc02f4343cfa144f0b9a3f8f553" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 35629 } } }
I20260810 06:43:09.908872 4664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:09.909687 4665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:09.913309 4677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:43:09.922381 4677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:43:09.939127 4677 catalog_manager.cc:1347] Generated new cluster ID: 06ce9b14d0a4452c93e5de4d8b460036
I20260810 06:43:09.939342 4677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:43:10.007411 4677 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:43:10.011029 4677 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:43:10.047441 4677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b60ebc02f4343cfa144f0b9a3f8f553: Generated new TSK 0
I20260810 06:43:10.049572 4677 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:43:10.073940 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 4539
W20260810 06:43:10.075703 4639 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55745
W20260810 06:43:10.076785 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 4539: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:43:10.077016 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 4539
2026-08-10T06:43:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2231 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:43:10 dist-test-slave-4tdj krb5kdc[4697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:43:10 dist-test-slave-4tdj krb5kdc[4697](info): set up 2 sockets
Aug 10 06:43:10 dist-test-slave-4tdj krb5kdc[4697](info): commencing operation
krb5kdc: starting...
W20260810 06:43:11.523931 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.385s user 0.001s sys 0.062s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:43:11 dist-test-slave-4tdj krb5kdc[4697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:43:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:43:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:43:11.791425 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:34509
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:33917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:34509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:12.115211 4727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:12.115485 4727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:12.115633 4727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:12.122534 4727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:12.122660 4727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:12.122738 4727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:12.122823 4727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:12.122885 4727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:12.132120 4727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:33917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:34509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:34509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:12.133293 4727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:12.134550 4727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:12.145955 4735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:12.146014 4734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:12.145994 4743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:12.146561 4727 server_base.cc:1034] running on GCE node
I20260810 06:43:12.147482 4727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:12.148864 4727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:12.150071 4727 hybrid_clock.cc:648] HybridClock initialized: now 1786344192150020 us; error 40 us; skew 500 ppm
I20260810 06:43:12.150432 4727 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:12 dist-test-slave-4tdj krb5kdc[4697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344192, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:12.157191 4727 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:43:12.158697 4727 webserver.cc:460] Webserver started at http://127.31.59.254:34447/ using document root <none> and password file <none>
I20260810 06:43:12.159351 4727 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:12.159471 4727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:12.159760 4727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:12.162729 4727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "a583440830b24bb5a836e0f92243c867"
format_stamp: "Formatted at 2026-08-10 06:43:12 on dist-test-slave-4tdj"
server_key: "4514f04bb4ed9ffb41b777cb13fb4a26"
server_key_iv: "3cf2c674e9fcc02954a539a83b56b1bd"
server_key_version: "encryptionkey@0"
I20260810 06:43:12.163466 4727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "a583440830b24bb5a836e0f92243c867"
format_stamp: "Formatted at 2026-08-10 06:43:12 on dist-test-slave-4tdj"
server_key: "4514f04bb4ed9ffb41b777cb13fb4a26"
server_key_iv: "3cf2c674e9fcc02954a539a83b56b1bd"
server_key_version: "encryptionkey@0"
I20260810 06:43:12.168552 4727 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:43:12.172025 4753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:12.173256 4727 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:43:12.173415 4727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "a583440830b24bb5a836e0f92243c867"
format_stamp: "Formatted at 2026-08-10 06:43:12 on dist-test-slave-4tdj"
server_key: "4514f04bb4ed9ffb41b777cb13fb4a26"
server_key_iv: "3cf2c674e9fcc02954a539a83b56b1bd"
server_key_version: "encryptionkey@0"
I20260810 06:43:12.173594 4727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:12.179136 4727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:12.182941 4727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:12.183198 4727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:12.200138 4727 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:34509
I20260810 06:43:12.200173 4829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:34509 every 8 connection(s)
I20260810 06:43:12.201401 4727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:43:12.204957 4830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:12.205417 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 4727
I20260810 06:43:12.205601 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:43:12.206003 31983 external_mini_cluster.cc:1428] Setting key 6f3eda619ec7b5d16b9d5de139d1600c
I20260810 06:43:12.215207 4835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:12.208365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57339 (local address 127.31.59.254:34509)
0810 06:43:12.209455 (+ 1090us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:12.209478 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:12.209529 (+ 51us) server_negotiation.cc:407] Connection header received
0810 06:43:12.211573 (+ 2044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:12.211604 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:12.212036 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:12.212470 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:12.213354 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:12.213362 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:12.213378 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:43:12.213393 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:12.213453 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:43:12.213487 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:12.213696 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:12.213822 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 06:43:12.214001 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":790,"thread_start_us":580,"threads_started":1}
I20260810 06:43:12.218662 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:43:12.218803 4830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867: Bootstrap starting.
I20260810 06:43:12.222555 4830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:12.223814 4830 log.cc:824] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:12.226065 4835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:12.223533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49015 (local address 127.31.59.254:34509)
0810 06:43:12.223729 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:12.223736 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:12.224210 (+ 474us) server_negotiation.cc:407] Connection header received
0810 06:43:12.224403 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:12.224408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:12.224474 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:12.224625 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:12.225499 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:12.225503 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:12.225506 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:43:12.225520 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:12.225578 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:43:12.225592 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:12.225672 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:12.225788 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 06:43:12.225827 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:43:12.226986 4830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867: No bootstrap required, opened a new log
I20260810 06:43:12.233942 4830 raft_consensus.cc:348] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a583440830b24bb5a836e0f92243c867" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34509 } }
I20260810 06:43:12.234272 4830 raft_consensus.cc:374] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:12.234409 4830 raft_consensus.cc:729] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a583440830b24bb5a836e0f92243c867, State: Initialized, Role: FOLLOWER
I20260810 06:43:12.234913 4830 consensus_queue.cc:260] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a583440830b24bb5a836e0f92243c867" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34509 } }
I20260810 06:43:12.235136 4830 raft_consensus.cc:388] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:43:12.235261 4830 raft_consensus.cc:482] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:43:12.235409 4830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:12.237589 4830 raft_consensus.cc:504] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a583440830b24bb5a836e0f92243c867" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34509 } }
I20260810 06:43:12.238044 4830 leader_election.cc:302] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a583440830b24bb5a836e0f92243c867; no voters:
I20260810 06:43:12.238660 4830 leader_election.cc:288] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:43:12.238745 4852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:12.239717 4852 raft_consensus.cc:686] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867 [term 1 LEADER]: Becoming Leader. State: Replica: a583440830b24bb5a836e0f92243c867, State: Running, Role: LEADER
I20260810 06:43:12.240193 4852 consensus_queue.cc:237] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a583440830b24bb5a836e0f92243c867" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34509 } }
I20260810 06:43:12.240495 4830 sys_catalog.cc:564] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:43:12.242758 4855 sys_catalog.cc:455] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a583440830b24bb5a836e0f92243c867. Latest consensus state: current_term: 1 leader_uuid: "a583440830b24bb5a836e0f92243c867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a583440830b24bb5a836e0f92243c867" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34509 } } }
I20260810 06:43:12.243165 4855 sys_catalog.cc:458] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:12.243671 4854 sys_catalog.cc:455] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a583440830b24bb5a836e0f92243c867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a583440830b24bb5a836e0f92243c867" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 34509 } } }
I20260810 06:43:12.243932 4854 sys_catalog.cc:458] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:12.245164 4860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:43:12.249220 4860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:43:12.257824 4860 catalog_manager.cc:1347] Generated new cluster ID: 35626aef81eb4cadbe7cf3fba83da6ac
I20260810 06:43:12.257917 4860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:43:12.267648 4860 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:43:12.268007 4860 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:43:12.279601 4860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a583440830b24bb5a836e0f92243c867: Generated new TSK 0
I20260810 06:43:12.280267 4860 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:43:12.291930 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 4727
W20260810 06:43:12.293268 4822 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57339
W20260810 06:43:12.293952 31983 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 4727: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:43:12.294085 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 4727
2026-08-10T06:43:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2196 ms)
[----------] 18 tests from AuthTokenIssuingTest (37812 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:43:12 dist-test-slave-4tdj krb5kdc[4886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:43:12 dist-test-slave-4tdj krb5kdc[4886](info): set up 2 sockets
Aug 10 06:43:12 dist-test-slave-4tdj krb5kdc[4886](info): commencing operation
krb5kdc: starting...
W20260810 06:43:13.537077 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.006s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:43:13 dist-test-slave-4tdj krb5kdc[4886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:43:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:43:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:43:13.816749 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:33025
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:45421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:33025,127.31.59.253:35017,127.31.59.252:42429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:14.154935 4902 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:43:14.155279 4902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:14.155395 4902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:14.155540 4902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:14.162811 4902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:14.162912 4902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:14.162995 4902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:14.163079 4902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:14.163177 4902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:14.173086 4902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:45421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:33025,127.31.59.253:35017,127.31.59.252:42429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:33025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:14.174401 4902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:14.175817 4902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:14.187656 4913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:14.187614 4914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:14.187682 4916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:14.188457 4902 server_base.cc:1034] running on GCE node
I20260810 06:43:14.189280 4902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:14.190754 4902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:14.191990 4902 hybrid_clock.cc:648] HybridClock initialized: now 1786344194191920 us; error 30 us; skew 500 ppm
I20260810 06:43:14.192297 4902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:14 dist-test-slave-4tdj krb5kdc[4886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344194, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:14.195439 4902 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:43:14.196798 4902 webserver.cc:460] Webserver started at http://127.31.59.254:36941/ using document root <none> and password file <none>
I20260810 06:43:14.197430 4902 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:14.197517 4902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:14.197781 4902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:14.200627 4902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "bb7c620e25df40e1936a9167e776ef46"
format_stamp: "Formatted at 2026-08-10 06:43:14 on dist-test-slave-4tdj"
server_key: "797bca0a319737981f98a68b140fd311"
server_key_iv: "c3beb6ea879b0cffbbd354ab7ac9ce55"
server_key_version: "encryptionkey@0"
I20260810 06:43:14.201349 4902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "bb7c620e25df40e1936a9167e776ef46"
format_stamp: "Formatted at 2026-08-10 06:43:14 on dist-test-slave-4tdj"
server_key: "797bca0a319737981f98a68b140fd311"
server_key_iv: "c3beb6ea879b0cffbbd354ab7ac9ce55"
server_key_version: "encryptionkey@0"
I20260810 06:43:14.206205 4902 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260810 06:43:14.209486 4935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:14.210934 4902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:43:14.211098 4902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "bb7c620e25df40e1936a9167e776ef46"
format_stamp: "Formatted at 2026-08-10 06:43:14 on dist-test-slave-4tdj"
server_key: "797bca0a319737981f98a68b140fd311"
server_key_iv: "c3beb6ea879b0cffbbd354ab7ac9ce55"
server_key_version: "encryptionkey@0"
I20260810 06:43:14.211294 4902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:14.237649 4902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:14.242412 4902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:14.242730 4902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:14.261497 4902 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:33025
I20260810 06:43:14.261518 5044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:33025 every 8 connection(s)
I20260810 06:43:14.263008 4902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:43:14.266820 5046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:14.272898 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 4902
I20260810 06:43:14.273105 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:43:14.273733 31983 external_mini_cluster.cc:1428] Setting key 5351e0201bbd1db235b28ca13e25f93b
I20260810 06:43:14.273314 5046 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:43:14 dist-test-slave-4tdj krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344194, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:43:14.295980 5049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.276480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34294 (local address 127.31.59.254:33025)
0810 06:43:14.278318 (+ 1838us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:14.278326 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:14.278377 (+ 51us) server_negotiation.cc:407] Connection header received
0810 06:43:14.280106 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:14.280111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:14.280327 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:14.280480 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:14.281924 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:14.283160 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:14.284925 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:14.285471 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:14.285601 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:14.289153 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:14.289160 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:14.289171 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:14.289198 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:14.291934 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:14.292560 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:14.292565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:14.292568 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:14.292703 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:14.293079 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:14.293082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:14.293084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:14.293545 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:14.293727 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:14.294251 (+ 524us) server_negotiation.cc:299] Negotiation successful
0810 06:43:14.294543 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1626,"spinlock_wait_cycles":41344,"thread_start_us":1382,"threads_started":1}
I20260810 06:43:14.296017 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.276480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:33025 (local address 127.0.0.1:34294)
0810 06:43:14.277368 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.277417 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:14.279731 (+ 2314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:14.280571 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:14.280578 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:14.281271 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:14.281724 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:14.281747 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:14.283305 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:14.283310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:14.284782 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:14.284797 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:14.285713 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:14.285720 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:14.285881 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:14.286548 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:14.286575 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:14.288941 (+ 2366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:14.292056 (+ 3115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:14.292068 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:14.292080 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:14.292438 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:14.292803 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:14.292808 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:14.292812 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:14.292974 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:14.293672 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:14.293680 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:14.293867 (+ 187us) client_negotiation.cc:769] Sending connection context
0810 06:43:14.294093 (+ 226us) client_negotiation.cc:240] Negotiation successful
0810 06:43:14.294701 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"mutex_wait_us":137,"thread_start_us":360,"threads_started":1}
I20260810 06:43:14.299077 5046 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:43:14.300055 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.299694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:35017 (local address 127.0.0.1:60706)
0810 06:43:14.299851 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.299947 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:35017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 06:43:14.300314 4941 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.59.253:35017: connect: Connection refused (error 111)
I20260810 06:43:14.301823 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.301490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:35017 (local address 127.0.0.1:60720)
0810 06:43:14.301631 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.301698 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:35017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 06:43:14.302019 5046 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.253:35017: Network error: Client connection negotiation failed: client connection to 127.31.59.253:35017: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.31.59.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.253.keytab.
Entry for principal kudu/127.31.59.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.253.keytab.
I20260810 06:43:14.338975 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.253:35017
--webserver_interface=127.31.59.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.253.keytab
--principal=kudu/127.31.59.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:45421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:33025,127.31.59.253:35017,127.31.59.252:42429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:43:14.351328 5046 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } attempt: 1
I20260810 06:43:14.352519 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.352061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:35017 (local address 127.0.0.1:60728)
0810 06:43:14.352252 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.352373 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:35017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 06:43:14.353487 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.353145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:35017 (local address 127.0.0.1:60742)
0810 06:43:14.353285 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.353358 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:35017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 06:43:14.353658 5046 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.253:35017: Network error: Client connection negotiation failed: client connection to 127.31.59.253:35017: connect: Connection refused (error 111)
I20260810 06:43:14.421896 5046 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } attempt: 2
I20260810 06:43:14.423179 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.422577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:35017 (local address 127.0.0.1:60746)
0810 06:43:14.422870 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.422994 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:35017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260810 06:43:14.424261 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.423828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:35017 (local address 127.0.0.1:60750)
0810 06:43:14.423986 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.424092 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:35017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 06:43:14.424482 5046 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.253:35017: Network error: Client connection negotiation failed: client connection to 127.31.59.253:35017: connect: Connection refused (error 111)
I20260810 06:43:14.515707 5046 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } attempt: 3
I20260810 06:43:14.516856 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.516335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:35017 (local address 127.0.0.1:60754)
0810 06:43:14.516569 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.516688 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:35017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260810 06:43:14.517937 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.517504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:35017 (local address 127.0.0.1:60762)
0810 06:43:14.517674 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.517767 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:35017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 06:43:14.518151 5046 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.253:35017: Network error: Client connection negotiation failed: client connection to 127.31.59.253:35017: connect: Connection refused (error 111)
I20260810 06:43:14.661396 5046 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } attempt: 4
I20260810 06:43:14.662788 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.662149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:35017 (local address 127.0.0.1:60776)
0810 06:43:14.662503 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.662622 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:35017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
I20260810 06:43:14.663782 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.663384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:35017 (local address 127.0.0.1:60792)
0810 06:43:14.663547 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.663642 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:35017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 06:43:14.663988 5046 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.253:35017: Network error: Client connection negotiation failed: client connection to 127.31.59.253:35017: connect: Connection refused (error 111)
W20260810 06:43:14.669166 5053 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:43:14.669463 5053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:14.669549 5053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:14.669663 5053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:14.676738 5053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:14.676815 5053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:14.676900 5053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:14.676959 5053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:14.677052 5053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:14.686842 5053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:45421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:33025,127.31.59.253:35017,127.31.59.252:42429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.253.keytab
--principal=kudu/127.31.59.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.253:35017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.59.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:14.688061 5053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:14.689340 5053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:14.700358 5059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:14.700395 5063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:14.700424 5067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:14.701090 5053 server_base.cc:1034] running on GCE node
I20260810 06:43:14.701767 5053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:14.703080 5053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:14.704294 5053 hybrid_clock.cc:648] HybridClock initialized: now 1786344194704228 us; error 44 us; skew 500 ppm
I20260810 06:43:14.704597 5053 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:14 dist-test-slave-4tdj krb5kdc[4886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344194, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:14.707690 5053 init.cc:377] Logged in from keytab as kudu/127.31.59.253@KRBTEST.COM (short username kudu)
I20260810 06:43:14.709167 5053 webserver.cc:460] Webserver started at http://127.31.59.253:41869/ using document root <none> and password file <none>
I20260810 06:43:14.709803 5053 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:14.709908 5053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:14.710203 5053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:14.713058 5053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-1/data/instance:
uuid: "f0a7af5fe11e431c942d1125ec9dece5"
format_stamp: "Formatted at 2026-08-10 06:43:14 on dist-test-slave-4tdj"
server_key: "16cb85810a71b909bc232e65f6a84d81"
server_key_iv: "034cd2fcf15e8118a2776e72e279f2fa"
server_key_version: "encryptionkey@0"
I20260810 06:43:14.713801 5053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-1/wal/instance:
uuid: "f0a7af5fe11e431c942d1125ec9dece5"
format_stamp: "Formatted at 2026-08-10 06:43:14 on dist-test-slave-4tdj"
server_key: "16cb85810a71b909bc232e65f6a84d81"
server_key_iv: "034cd2fcf15e8118a2776e72e279f2fa"
server_key_version: "encryptionkey@0"
I20260810 06:43:14.718923 5053 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 06:43:14.722265 5080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:14.723516 5053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:43:14.723690 5053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-1/wal
uuid: "f0a7af5fe11e431c942d1125ec9dece5"
format_stamp: "Formatted at 2026-08-10 06:43:14 on dist-test-slave-4tdj"
server_key: "16cb85810a71b909bc232e65f6a84d81"
server_key_iv: "034cd2fcf15e8118a2776e72e279f2fa"
server_key_version: "encryptionkey@0"
I20260810 06:43:14.723884 5053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:14.735666 5053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:14.739475 5053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:14.739719 5053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:14.755559 5053 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.253:35017
I20260810 06:43:14.755578 5177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.253:35017 every 8 connection(s)
I20260810 06:43:14.756855 5053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-1/data/info.pb
I20260810 06:43:14.757871 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 5053
I20260810 06:43:14.758040 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-1/wal/instance
I20260810 06:43:14.758522 31983 external_mini_cluster.cc:1428] Setting key 3ce1afab205b93239609044fdc8267ab
I20260810 06:43:14.760803 5182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:14.767143 5182 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:43:14 dist-test-slave-4tdj krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344194, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.253@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:43:14.786583 5049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.769623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34298 (local address 127.31.59.254:33025)
0810 06:43:14.769787 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:14.769791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:14.771062 (+ 1271us) server_negotiation.cc:407] Connection header received
0810 06:43:14.773273 (+ 2211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:14.773279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:14.773362 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:14.773449 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:14.774917 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:14.775586 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:14.777050 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:14.777449 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:14.777558 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:14.781036 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:14.781042 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:14.781045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:14.781079 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:14.783650 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:14.784336 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:14.784340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:14.784342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:14.784473 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:14.784873 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:14.784877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:14.784879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:14.785153 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:14.785310 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:14.785991 (+ 681us) server_negotiation.cc:299] Negotiation successful
0810 06:43:14.786221 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
WARNING: no policy specified for kudu/127.31.59.252@KRBTEST.COM; defaulting to no policy
I20260810 06:43:14.787479 5188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.770135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:33025 (local address 127.0.0.1:34298)
0810 06:43:14.770896 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.770938 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:14.773014 (+ 2076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:14.773685 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:14.773693 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:14.774175 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:14.774750 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:14.774767 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:14.775736 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:14.775740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:14.776895 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:14.776907 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:14.777671 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:14.777679 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:14.777884 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:14.778556 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:14.778578 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:14.780830 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:14.783780 (+ 2950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:14.783788 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:14.783804 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:14.784215 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:14.784575 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:14.784580 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:14.784584 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:14.784763 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:14.785284 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:14.785291 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:14.785636 (+ 345us) client_negotiation.cc:769] Sending connection context
0810 06:43:14.785860 (+ 224us) client_negotiation.cc:240] Negotiation successful
0810 06:43:14.786375 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":345,"threads_started":1}
I20260810 06:43:14.789287 5182 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.252@KRBTEST.COM" created.
Aug 10 06:43:14 dist-test-slave-4tdj krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344194, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.253@KRBTEST.COM for kudu/127.31.59.253@KRBTEST.COM
I20260810 06:43:14.803625 5188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.789898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:35017 (local address 127.0.0.1:60800)
0810 06:43:14.790094 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.790112 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:14.790193 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:14.791296 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:14.791301 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:14.791738 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:14.791912 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:14.791924 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:14.793300 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:14.793305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:14.794423 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:14.794437 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:14.795247 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:14.795252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:14.795395 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:14.795940 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:14.795958 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:14.798242 (+ 2284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:14.801233 (+ 2991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:14.801238 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:14.801241 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:14.801670 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:14.802059 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:14.802062 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:14.802064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:14.802191 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:14.802834 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:14.802838 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:14.802946 (+ 108us) client_negotiation.cc:769] Sending connection context
0810 06:43:14.803066 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 06:43:14.803257 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 06:43:14.803753 5191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.790239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60800 (local address 127.31.59.253:35017)
0810 06:43:14.790879 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:14.790885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:14.790909 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:43:14.790951 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:14.790957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:14.791135 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:14.791247 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:14.792099 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:14.793166 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:14.794550 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:14.795089 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:14.795183 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:14.798454 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:14.798465 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:14.798478 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:14.798506 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:14.801116 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:14.801800 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:14.801809 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:14.801813 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:14.801959 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:14.802303 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:14.802308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:14.802311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:14.802716 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:14.802918 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:14.803217 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 06:43:14.803397 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":369,"threads_started":1}
I20260810 06:43:14.805801 5182 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:43:14.807015 5188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.806550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58470)
0810 06:43:14.806726 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.806853 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 06:43:14.807281 5082 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
I20260810 06:43:14.808780 5188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.808419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58474)
0810 06:43:14.808580 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.808670 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 06:43:14.808992 5182 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:42429: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.252.keytab.
Entry for principal kudu/127.31.59.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.252.keytab.
I20260810 06:43:14.825752 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.252:42429
--webserver_interface=127.31.59.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.252.keytab
--principal=kudu/127.31.59.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:45421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:33025,127.31.59.253:35017,127.31.59.252:42429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:43:14.858350 5182 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } attempt: 1
I20260810 06:43:14.859643 5188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.859093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58476)
0810 06:43:14.859318 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.859449 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 06:43:14.860663 5188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.860266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58488)
0810 06:43:14.860432 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.860523 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 06:43:14.860857 5182 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:42429: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
I20260810 06:43:14.929069 5182 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } attempt: 2
I20260810 06:43:14.930341 5188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.929756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58498)
0810 06:43:14.929980 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.930103 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 06:43:14.931422 5188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.931024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58508)
0810 06:43:14.931182 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.931281 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 06:43:14.931669 5182 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:42429: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
I20260810 06:43:14.963246 5046 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } attempt: 5
Aug 10 06:43:14 dist-test-slave-4tdj krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344194, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for kudu/127.31.59.253@KRBTEST.COM
I20260810 06:43:14.977319 5191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.964246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60806 (local address 127.31.59.253:35017)
0810 06:43:14.964453 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:14.964459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:14.964479 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:43:14.964536 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:14.964540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:14.964617 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:14.964703 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:14.965553 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:14.966259 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:14.967973 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:14.968437 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:14.968537 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:14.972291 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:14.972296 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:14.972299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:14.972330 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:14.974798 (+ 2468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:14.975556 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:14.975560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:14.975562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:14.975684 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:14.976078 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:14.976081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:14.976083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:14.976352 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:14.976471 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:14.976908 (+ 437us) server_negotiation.cc:299] Negotiation successful
0810 06:43:14.977098 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 06:43:14.977449 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.964003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:35017 (local address 127.0.0.1:60806)
0810 06:43:14.964204 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.964226 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:14.964359 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:14.964773 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:14.964778 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:14.965184 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:14.965387 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:14.965398 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:14.966445 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:14.966452 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:14.967789 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:14.967808 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:14.968604 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:14.968612 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:14.968878 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:14.969565 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:14.969591 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:14.972099 (+ 2508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:14.974940 (+ 2841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:14.974946 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:14.974950 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:14.975444 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:14.975788 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:14.975793 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:14.975797 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:14.975964 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:14.976473 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:14.976478 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:14.976662 (+ 184us) client_negotiation.cc:769] Sending connection context
0810 06:43:14.976816 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 06:43:14.977109 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 06:43:14.978664 5046 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:43:14.979728 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.979286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58512)
0810 06:43:14.979464 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.979560 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 06:43:14.980751 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:14.980364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58528)
0810 06:43:14.980525 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:14.980623 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 06:43:14.981002 5046 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:42429: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
I20260810 06:43:15.022897 5182 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } attempt: 3
I20260810 06:43:15.024410 5188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.023673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58538)
0810 06:43:15.024050 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:15.024193 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260810 06:43:15.026398 5188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.025724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58554)
0810 06:43:15.026101 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:15.026215 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
W20260810 06:43:15.026844 5182 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:42429: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
I20260810 06:43:15.032236 5046 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } attempt: 1
I20260810 06:43:15.033282 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.032802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58562)
0810 06:43:15.032996 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:15.033096 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 06:43:15.034224 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.033859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58566)
0810 06:43:15.034012 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:15.034097 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 06:43:15.034469 5046 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:42429: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
I20260810 06:43:15.102702 5046 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } attempt: 2
I20260810 06:43:15.103948 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.103423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58582)
0810 06:43:15.103669 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:15.103766 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260810 06:43:15.104951 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.104648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58594)
0810 06:43:15.104772 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:15.104840 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 06:43:15.105168 5046 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:42429: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
W20260810 06:43:15.161463 5197 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:43:15.161767 5197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:15.161837 5197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:15.161916 5197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:15.168995 5197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:15.169082 5197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:15.169143 5197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:15.169193 5197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:15.169252 5197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:15.170065 5182 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } attempt: 4
I20260810 06:43:15.171267 5188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.170694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58598)
0810 06:43:15.170950 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:15.171077 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 06:43:15.172287 5188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.171912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58602)
0810 06:43:15.172070 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:15.172154 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 06:43:15.172533 5182 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:42429: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
I20260810 06:43:15.179303 5197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:45421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:33025,127.31.59.253:35017,127.31.59.252:42429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.252.keytab
--principal=kudu/127.31.59.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.252:42429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.59.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:15.180526 5197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:15.181808 5197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:15.193059 5209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:15.193104 5211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:15.193176 5213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:15.193759 5197 server_base.cc:1034] running on GCE node
I20260810 06:43:15.194550 5197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:15.195863 5197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:15.197089 5197 hybrid_clock.cc:648] HybridClock initialized: now 1786344195197021 us; error 39 us; skew 500 ppm
I20260810 06:43:15.197393 5197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:15 dist-test-slave-4tdj krb5kdc[4886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344195, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:15.200471 5197 init.cc:377] Logged in from keytab as kudu/127.31.59.252@KRBTEST.COM (short username kudu)
I20260810 06:43:15.201833 5197 webserver.cc:460] Webserver started at http://127.31.59.252:44349/ using document root <none> and password file <none>
I20260810 06:43:15.202559 5197 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:15.202661 5197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:15.202962 5197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:15.205822 5197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-2/data/instance:
uuid: "fc3e231ddf064f4cb65987eddae82d10"
format_stamp: "Formatted at 2026-08-10 06:43:15 on dist-test-slave-4tdj"
server_key: "f2f2576e4468c26b62dccb3023ce7881"
server_key_iv: "547f210a0ee8078df10245052b3dc9b3"
server_key_version: "encryptionkey@0"
I20260810 06:43:15.206709 5197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-2/wal/instance:
uuid: "fc3e231ddf064f4cb65987eddae82d10"
format_stamp: "Formatted at 2026-08-10 06:43:15 on dist-test-slave-4tdj"
server_key: "f2f2576e4468c26b62dccb3023ce7881"
server_key_iv: "547f210a0ee8078df10245052b3dc9b3"
server_key_version: "encryptionkey@0"
I20260810 06:43:15.211369 5046 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } attempt: 3
I20260810 06:43:15.211809 5197 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:43:15.212445 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.211877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58606)
0810 06:43:15.212137 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:15.212246 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 06:43:15.213577 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.213141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58612)
0810 06:43:15.213357 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:15.213446 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260810 06:43:15.213785 5046 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:42429: Network error: Client connection negotiation failed: client connection to 127.31.59.252:42429: connect: Connection refused (error 111)
I20260810 06:43:15.215785 5222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:15.217151 5197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:43:15.217326 5197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-2/wal
uuid: "fc3e231ddf064f4cb65987eddae82d10"
format_stamp: "Formatted at 2026-08-10 06:43:15 on dist-test-slave-4tdj"
server_key: "f2f2576e4468c26b62dccb3023ce7881"
server_key_iv: "547f210a0ee8078df10245052b3dc9b3"
server_key_version: "encryptionkey@0"
I20260810 06:43:15.217554 5197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:15.236321 5197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:15.240819 5197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:15.241119 5197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:15.259876 5197 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.252:42429
I20260810 06:43:15.259896 5300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.252:42429 every 8 connection(s)
I20260810 06:43:15.261325 5197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-2/data/info.pb
I20260810 06:43:15.263689 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 5197
I20260810 06:43:15.263862 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343831713822-31983-0/minicluster-data/master-2/wal/instance
I20260810 06:43:15.264305 31983 external_mini_cluster.cc:1428] Setting key d8d87d446e42e84148f6e11a09e452ab
I20260810 06:43:15.265859 5308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:15.275010 5308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:43:15 dist-test-slave-4tdj krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.252@KRBTEST.COM
Aug 10 06:43:15 dist-test-slave-4tdj krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344195, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.252@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:43:15.288121 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.266146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58626 (local address 127.31.59.252:42429)
0810 06:43:15.266992 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.267006 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.267054 (+ 48us) server_negotiation.cc:407] Connection header received
0810 06:43:15.269255 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.269283 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.269786 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.270205 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.272008 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.273352 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.275064 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.275734 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.275890 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:15.280137 (+ 4247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:15.280143 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:15.280153 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:15.280187 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:15.283480 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.284281 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.284291 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.284298 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.284503 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.284935 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.284941 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.284944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.285579 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:15.285809 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:15.286272 (+ 463us) server_negotiation.cc:299] Negotiation successful
0810 06:43:15.286747 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":358,"threads_started":1}
I20260810 06:43:15.291787 5049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.277668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34306 (local address 127.31.59.254:33025)
0810 06:43:15.277870 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.277877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.278533 (+ 656us) server_negotiation.cc:407] Connection header received
0810 06:43:15.278711 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.278717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.278798 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.278896 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.279908 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.280598 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.281981 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.282653 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.282782 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:15.286198 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:15.286205 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:15.286210 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:15.286250 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:15.289118 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.289798 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.289803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.289806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.289945 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.290281 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.290285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.290287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.290578 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:15.290719 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:15.291275 (+ 556us) server_negotiation.cc:299] Negotiation successful
0810 06:43:15.291519 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:43:15.292557 5312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.277642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:33025 (local address 127.0.0.1:34306)
0810 06:43:15.278397 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:15.278445 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:15.278593 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:15.278940 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:15.278945 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:15.279533 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:15.279768 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:15.279777 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.280731 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.280735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:15.281836 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:15.281848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.282832 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.282837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:15.282957 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:15.283519 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:15.283539 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:15.286016 (+ 2477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:15.289238 (+ 3222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:15.289249 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:15.289255 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:15.289676 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:15.290054 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:15.290059 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:15.290061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:15.290181 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:15.290709 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:15.290716 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:15.290917 (+ 201us) client_negotiation.cc:769] Sending connection context
0810 06:43:15.292011 (+ 1094us) client_negotiation.cc:240] Negotiation successful
0810 06:43:15.292278 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":361,"threads_started":1}
I20260810 06:43:15.293895 5308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:43:15 dist-test-slave-4tdj krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
Aug 10 06:43:15 dist-test-slave-4tdj krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344195, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.252@KRBTEST.COM for kudu/127.31.59.253@KRBTEST.COM
I20260810 06:43:15.306525 5313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.290911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34312 (local address 127.31.59.254:33025)
0810 06:43:15.291648 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.291653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.291675 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:43:15.291713 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.291716 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.291766 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.291857 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.293227 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.294261 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.295733 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.296242 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.296332 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:15.300667 (+ 4335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:15.300673 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:15.300678 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:15.300716 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:15.303469 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.304195 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.304202 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.304206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.304362 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.304819 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.304825 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.304828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.305266 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:15.305479 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:15.305821 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 06:43:15.306064 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":419,"threads_started":1}
I20260810 06:43:15.306571 5191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.294653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60808 (local address 127.31.59.253:35017)
0810 06:43:15.294842 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.294848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.294867 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:43:15.294923 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.294928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.295003 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.295128 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.295696 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.296407 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.297523 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.298168 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.298315 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:15.301220 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:15.301225 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:15.301228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:15.301258 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:15.303780 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.304552 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.304557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.304560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.304717 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.305150 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.305153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.305155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.305451 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:15.305603 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:15.306048 (+ 445us) server_negotiation.cc:299] Negotiation successful
0810 06:43:15.306267 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:43:15.306581 5312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.294497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:35017 (local address 127.0.0.1:60808)
0810 06:43:15.294719 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:15.294736 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:15.294816 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:15.295129 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:15.295132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:15.295430 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:15.295561 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:15.295568 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.296523 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.296526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:15.297385 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:15.297396 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.298341 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.298348 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:15.298478 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:15.298945 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:15.298960 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:15.301047 (+ 2087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:15.303924 (+ 2877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:15.303928 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:15.303931 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:15.304409 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:15.304826 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:15.304831 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:15.304834 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:15.305044 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:15.305588 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:15.305592 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:15.305763 (+ 171us) client_negotiation.cc:769] Sending connection context
0810 06:43:15.305925 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 06:43:15.306181 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132}
I20260810 06:43:15.307560 5308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:43:15 dist-test-slave-4tdj krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344195, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.252@KRBTEST.COM for kudu/127.31.59.252@KRBTEST.COM
Aug 10 06:43:15 dist-test-slave-4tdj krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.253@KRBTEST.COM
I20260810 06:43:15.321442 5312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.308252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58634)
0810 06:43:15.308408 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:15.308427 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:15.308525 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:15.308937 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:15.308941 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:15.309407 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:15.309582 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:15.309594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.310773 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.310777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:15.311883 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:15.311898 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.312634 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.312639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:15.312820 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:15.313375 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:15.313396 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:15.315714 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:15.319044 (+ 3330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:15.319049 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:15.319064 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:15.319458 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:15.319867 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:15.319871 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:15.319873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:15.319995 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:15.320565 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:15.320569 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:15.320704 (+ 135us) client_negotiation.cc:769] Sending connection context
0810 06:43:15.320831 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 06:43:15.321048 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":31}
I20260810 06:43:15.321599 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.308516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58634 (local address 127.31.59.252:42429)
0810 06:43:15.308670 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.308676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.308693 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:43:15.308736 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.308741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.308810 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.308913 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.309734 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.310652 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.312027 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.312443 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.312618 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:15.315907 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:15.315911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:15.315915 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:15.315943 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:15.318898 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.319582 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.319588 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.319591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.319748 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.320118 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.320123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.320126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.320458 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:15.320606 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:15.320951 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 06:43:15.321220 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":55}
I20260810 06:43:15.323850 5191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.308357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60820 (local address 127.31.59.253:35017)
0810 06:43:15.308522 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.308527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.308580 (+ 53us) server_negotiation.cc:407] Connection header received
0810 06:43:15.308736 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.308741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.308810 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.308913 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.310226 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.311120 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.312539 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.313112 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.313241 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:15.317974 (+ 4733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:15.317980 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:15.317985 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:15.318037 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:15.321089 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.321904 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.321908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.321910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.322047 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.322536 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.322541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.322544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.322945 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:15.323177 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:15.323438 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 06:43:15.323630 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:43:15.327503 5308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10: Bootstrap starting.
I20260810 06:43:15.328469 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:43:15.330937 5308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:15.332280 5308 log.cc:824] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:15.331141 4992 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "bb7c620e25df40e1936a9167e776ef46"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34312
I20260810 06:43:15.335459 5308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10: No bootstrap required, opened a new log
I20260810 06:43:15.344666 5308 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } } peers { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } } peers { permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } }
I20260810 06:43:15.344980 5308 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3e231ddf064f4cb65987eddae82d10, State: Initialized, Role: FOLLOWER
I20260810 06:43:15.345376 5308 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } } peers { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } } peers { permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } }
I20260810 06:43:15.346184 5318 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } } peers { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } } peers { permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } } }
I20260810 06:43:15.346531 5318 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:43:15.347259 5308 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 06:43:15.356206 5332 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:43:15.356329 5332 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:43:15.354413 5197 master_runner.cc:186] Error getting master registration for 127.31.59.254:33025: OK, instance_id { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" instance_seqno: 1786344194256037 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:43:15.364952 5197 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:43:15.391002 5046 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } attempt: 4
Aug 10 06:43:15 dist-test-slave-4tdj krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344194, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for kudu/127.31.59.252@KRBTEST.COM
I20260810 06:43:15.406744 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.391706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58640)
0810 06:43:15.391915 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:15.391937 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:15.392080 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:15.392411 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:15.392417 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:15.392922 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:15.393154 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:15.393167 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.394573 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.394579 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:15.396046 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:15.396061 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.397062 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.397069 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:15.397271 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:15.397918 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:15.397942 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:15.400527 (+ 2585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:15.403995 (+ 3468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:15.404001 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:15.404005 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:15.404519 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:15.404892 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:15.404897 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:15.404901 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:15.405107 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:15.405726 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:15.405731 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:15.405891 (+ 160us) client_negotiation.cc:769] Sending connection context
0810 06:43:15.406052 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 06:43:15.406357 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 06:43:15.406750 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.391768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58640 (local address 127.31.59.252:42429)
0810 06:43:15.391970 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.391974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.391993 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:43:15.392211 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.392215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.392278 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.392358 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.393300 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.394444 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.396185 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.396909 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.397020 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:15.400710 (+ 3690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:15.400714 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:15.400718 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:15.400750 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:15.403857 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.404633 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.404639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.404643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.404785 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.405223 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.405228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.405232 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.405612 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:15.405760 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:15.406148 (+ 388us) server_negotiation.cc:299] Negotiation successful
0810 06:43:15.406396 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:43:15.412678 5046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46: Bootstrap starting.
I20260810 06:43:15.416769 5046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:15.418076 5046 log.cc:824] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:15.421430 5046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46: No bootstrap required, opened a new log
I20260810 06:43:15.428933 5046 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } } peers { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } } peers { permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } }
I20260810 06:43:15.429342 5046 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb7c620e25df40e1936a9167e776ef46, State: Initialized, Role: FOLLOWER
I20260810 06:43:15.429849 5046 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } } peers { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } } peers { permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } }
I20260810 06:43:15.430819 5338 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } } peers { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } } peers { permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } } }
I20260810 06:43:15.431187 5338 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:43:15.432051 5046 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:43:15.432214 4992 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 06:43:15.432394 4992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:15.434834 4992 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } } peers { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } } peers { permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } }
I20260810 06:43:15.436807 5258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bb7c620e25df40e1936a9167e776ef46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fc3e231ddf064f4cb65987eddae82d10"
I20260810 06:43:15.437306 5258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:15.440671 5258 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb7c620e25df40e1936a9167e776ef46 in term 1.
I20260810 06:43:15.441063 4992 leader_election.cc:288] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [CANDIDATE]: Term 1 election: Requested vote from peers f0a7af5fe11e431c942d1125ec9dece5 (127.31.59.253:35017), fc3e231ddf064f4cb65987eddae82d10 (127.31.59.252:42429)
I20260810 06:43:15.441383 4937 leader_election.cc:302] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb7c620e25df40e1936a9167e776ef46, fc3e231ddf064f4cb65987eddae82d10; no voters:
I20260810 06:43:15.441846 5338 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:15.456873 5338 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [term 1 LEADER]: Becoming Leader. State: Replica: bb7c620e25df40e1936a9167e776ef46, State: Running, Role: LEADER
I20260810 06:43:15.457585 5338 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } } peers { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } } peers { permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } }
I20260810 06:43:15.471732 5182 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } attempt: 5
Aug 10 06:43:15 dist-test-slave-4tdj krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344194, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.253@KRBTEST.COM for kudu/127.31.59.252@KRBTEST.COM
I20260810 06:43:15.491346 5188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.472510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:42429 (local address 127.0.0.1:58648)
0810 06:43:15.472962 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:15.472984 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:15.473121 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:15.474070 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:15.474077 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:15.474672 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:15.474921 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:15.474935 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.476518 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.476524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:15.477904 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:15.477919 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.479119 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.479125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:15.479371 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:15.480154 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:15.480181 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:15.484425 (+ 4244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:15.488431 (+ 4006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:15.488436 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:15.488439 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:15.489003 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:15.489462 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:15.489467 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:15.489471 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:15.489644 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:15.490334 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:15.490339 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:15.490499 (+ 160us) client_negotiation.cc:769] Sending connection context
0810 06:43:15.490659 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 06:43:15.490952 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305}
I20260810 06:43:15.492522 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.473369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58648 (local address 127.31.59.252:42429)
0810 06:43:15.473558 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.473564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.473583 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:43:15.473646 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.473651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.473738 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.473840 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.475146 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.476327 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.478082 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.478798 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.478921 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:15.484633 (+ 5712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:15.484639 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:15.484644 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:15.484684 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:15.488266 (+ 3582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.489156 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.489162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.489166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.489329 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.489770 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.489774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.489777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.490116 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:15.490245 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:15.491931 (+ 1686us) server_negotiation.cc:299] Negotiation successful
0810 06:43:15.492178 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 06:43:15.493330 5230 tablet.cc:2369] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:43:15.495607 4945 tablet.cc:2369] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:43:15.498782 5354 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb7c620e25df40e1936a9167e776ef46. Latest consensus state: current_term: 1 leader_uuid: "bb7c620e25df40e1936a9167e776ef46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } } peers { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } } peers { permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } } }
I20260810 06:43:15.499204 5354 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:15.504055 5182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5: Bootstrap starting.
I20260810 06:43:15.510124 5313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.483041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34316 (local address 127.31.59.254:33025)
0810 06:43:15.483260 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.483269 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.484254 (+ 985us) server_negotiation.cc:407] Connection header received
0810 06:43:15.484545 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.484551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.484633 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.484703 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.487105 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.488202 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.490005 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.490766 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.490856 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:15.496080 (+ 5224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:15.496087 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:15.496092 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:15.496137 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:15.503980 (+ 7843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.507468 (+ 3488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.507474 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.507477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.507744 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.508274 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.508279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.508283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.508672 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:15.508925 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:15.509499 (+ 574us) server_negotiation.cc:299] Negotiation successful
0810 06:43:15.509755 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:43:15.513537 5191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.494331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60830 (local address 127.31.59.253:35017)
0810 06:43:15.494504 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.494509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.496630 (+ 2121us) server_negotiation.cc:407] Connection header received
0810 06:43:15.496901 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.496906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.496981 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.497041 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.498897 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.499892 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.501651 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.502344 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.502465 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:15.505877 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:15.505883 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:15.505888 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:15.505933 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:15.509889 (+ 3956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.510731 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.510737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.510740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.510911 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.511406 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.511411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.511414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.511757 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:15.511984 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:15.512718 (+ 734us) server_negotiation.cc:299] Negotiation successful
0810 06:43:15.513148 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:43:15.511221 4902 master_runner.cc:186] Error getting master registration for 127.31.59.253:35017: OK, instance_id { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" instance_seqno: 1786344194751201 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:43:15.523427 4902 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:43:15.524324 5182 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5: Neither blocks nor log segments found. Creating new log.
W20260810 06:43:15.524830 5380 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:43:15.524961 5380 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:43:15.526069 5182 log.cc:824] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:15.526783 5381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:43:15.528506 5381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:43:15.531508 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:15.476970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58656 (local address 127.31.59.252:42429)
0810 06:43:15.478857 (+ 1887us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.478862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.478883 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:43:15.478923 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.478928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.478994 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.479110 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.482942 (+ 3832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.485125 (+ 2183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.488720 (+ 3595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:15.489966 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:15.492354 (+ 2388us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:15.519418 (+ 27064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:15.519426 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:15.519431 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:15.519479 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:15.524364 (+ 4885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.525965 (+ 1601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.525972 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.526003 (+ 31us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.526458 (+ 455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:15.527307 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:15.527313 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:15.527316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:15.528005 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:15.529972 (+ 1967us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:15.530235 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 06:43:15.530761 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1762,"thread_start_us":1558,"threads_started":1}
I20260810 06:43:15.537016 5182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5: No bootstrap required, opened a new log
I20260810 06:43:15.548616 5258 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10 [term 1 FOLLOWER]: Refusing update from remote peer bb7c620e25df40e1936a9167e776ef46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:15.549706 5354 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:43:15.553124 5182 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } } peers { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } } peers { permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } }
I20260810 06:43:15.553596 5182 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0a7af5fe11e431c942d1125ec9dece5, State: Initialized, Role: FOLLOWER
I20260810 06:43:15.554142 5182 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } } peers { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } } peers { permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } }
I20260810 06:43:15.557247 5182 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:43:15.558547 5134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bb7c620e25df40e1936a9167e776ef46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f0a7af5fe11e431c942d1125ec9dece5"
I20260810 06:43:15.559039 5134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:15.565670 5134 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb7c620e25df40e1936a9167e776ef46 in term 1.
I20260810 06:43:15.566431 5133 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5 [term 1 FOLLOWER]: Refusing update from remote peer bb7c620e25df40e1936a9167e776ef46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:15.568346 5354 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:43:15.583060 5318 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb7c620e25df40e1936a9167e776ef46. Latest consensus state: current_term: 1 leader_uuid: "bb7c620e25df40e1936a9167e776ef46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } } peers { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } } peers { permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } } }
I20260810 06:43:15.583377 5318 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:43:15.585237 5385 mvcc.cc:205] Tried to move back new op lower bound from 7316865824956076032 to 7316865824600449024. Current Snapshot: MvccSnapshot[applied={T|T < 7316865824956076032}]
I20260810 06:43:15.606420 5318 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bb7c620e25df40e1936a9167e776ef46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } } peers { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } } peers { permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } } }
I20260810 06:43:15.606676 5318 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:43:15.606462 5387 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb7c620e25df40e1936a9167e776ef46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } } peers { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } } peers { permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } } }
I20260810 06:43:15.606973 5387 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:43:15.607462 5354 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bb7c620e25df40e1936a9167e776ef46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } } peers { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } } peers { permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } } }
I20260810 06:43:15.607699 5354 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:15.607504 5391 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb7c620e25df40e1936a9167e776ef46. Latest consensus state: current_term: 1 leader_uuid: "bb7c620e25df40e1936a9167e776ef46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } } peers { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } } peers { permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } } }
I20260810 06:43:15.607869 5391 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:43:15.609637 5381 catalog_manager.cc:1347] Generated new cluster ID: 2cfaebb6f7534eac938afaa497b33762
I20260810 06:43:15.609741 5381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:43:15.612128 5338 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bb7c620e25df40e1936a9167e776ef46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } } peers { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } } peers { permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } } }
I20260810 06:43:15.612782 5338 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:15.618427 5394 mvcc.cc:205] Tried to move back new op lower bound from 7316865824956076032 to 7316865824600449024. Current Snapshot: MvccSnapshot[applied={T|T < 7316865824956076032}]
I20260810 06:43:15.621272 5391 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bb7c620e25df40e1936a9167e776ef46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb7c620e25df40e1936a9167e776ef46" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33025 } } peers { permanent_uuid: "f0a7af5fe11e431c942d1125ec9dece5" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 35017 } } peers { permanent_uuid: "fc3e231ddf064f4cb65987eddae82d10" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 42429 } } }
I20260810 06:43:15.621496 5391 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:43:15.640590 5415 catalog_manager.cc:1259] Loaded cluster ID: 2cfaebb6f7534eac938afaa497b33762
I20260810 06:43:15.640748 5415 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5: loading cluster ID for follower catalog manager: success
W20260810 06:43:15.641954 5415 catalog_manager.cc:1576] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260810 06:43:15.642138 5415 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:43:15.662463 5381 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:43:15.664299 5381 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:43:15.682070 5381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb7c620e25df40e1936a9167e776ef46: Generated new TSK 0
I20260810 06:43:15.683120 5381 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:43:15.826572 5313 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:15.806875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34318 (local address 127.31.59.254:33025)
0810 06:43:15.807090 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.807097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.824517 (+ 17420us) server_negotiation.cc:407] Connection header received
0810 06:43:15.824865 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.824872 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.824961 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.825041 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.826261 (+ 1220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34318: BlockingRecv error: recv got EOF from 127.0.0.1:34318 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260810 06:43:15.828729 5376 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:15.808766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58672 (local address 127.31.59.252:42429)
0810 06:43:15.808951 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.808956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.827162 (+ 18206us) server_negotiation.cc:407] Connection header received
0810 06:43:15.827387 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.827393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.827513 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.827578 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.828482 (+ 904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58672: BlockingRecv error: recv got EOF from 127.0.0.1:58672 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260810 06:43:15.832240 5191 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:15.829973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60834 (local address 127.31.59.253:35017)
0810 06:43:15.830170 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.830176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.830623 (+ 447us) server_negotiation.cc:407] Connection header received
0810 06:43:15.830840 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.830846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.830939 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.831007 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.832013 (+ 1006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60834: BlockingRecv error: recv got EOF from 127.0.0.1:60834 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260810 06:43:15.868988 5191 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:15.859197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60850 (local address 127.31.59.253:35017)
0810 06:43:15.859368 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.859374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.866927 (+ 7553us) server_negotiation.cc:407] Connection header received
0810 06:43:15.867420 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.867427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.867561 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.867637 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.868683 (+ 1046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60850: BlockingRecv error: recv got EOF from 127.0.0.1:60850 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 06:43:15.909058 5191 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:15.899655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60860 (local address 127.31.59.253:35017)
0810 06:43:15.899838 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.899844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.907129 (+ 7285us) server_negotiation.cc:407] Connection header received
0810 06:43:15.907612 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.907619 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.907708 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.907785 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.908781 (+ 996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60860: BlockingRecv error: recv got EOF from 127.0.0.1:60860 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 06:43:15.956741 5191 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:15.946754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60872 (local address 127.31.59.253:35017)
0810 06:43:15.946977 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.946983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.954757 (+ 7774us) server_negotiation.cc:407] Connection header received
0810 06:43:15.955146 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.955153 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.955274 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.955363 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.956447 (+ 1084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60872: BlockingRecv error: recv got EOF from 127.0.0.1:60872 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260810 06:43:15.982496 5191 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:15.980331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60876 (local address 127.31.59.253:35017)
0810 06:43:15.980634 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.980640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.980911 (+ 271us) server_negotiation.cc:407] Connection header received
0810 06:43:15.981146 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.981152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.981249 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.981363 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.982227 (+ 864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60876: BlockingRecv error: recv got EOF from 127.0.0.1:60876 (error 108)
Metrics: {"server-negotiator.queue_time_us":184}
W20260810 06:43:15.996517 5191 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:15.994218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60880 (local address 127.31.59.253:35017)
0810 06:43:15.994401 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:15.994410 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:15.994976 (+ 566us) server_negotiation.cc:407] Connection header received
0810 06:43:15.995186 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:15.995191 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:15.995278 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:15.995389 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:15.996255 (+ 866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60880: BlockingRecv error: recv got EOF from 127.0.0.1:60880 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 06:43:16.019153 5191 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:16.016751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60884 (local address 127.31.59.253:35017)
0810 06:43:16.016939 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:16.016946 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:16.017502 (+ 556us) server_negotiation.cc:407] Connection header received
0810 06:43:16.017745 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:16.017751 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:16.017858 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:16.017975 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:16.018888 (+ 913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60884: BlockingRecv error: recv got EOF from 127.0.0.1:60884 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260810 06:43:16.057614 5191 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:16.055241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60894 (local address 127.31.59.253:35017)
0810 06:43:16.055440 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:16.055445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:16.056007 (+ 562us) server_negotiation.cc:407] Connection header received
0810 06:43:16.056222 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:16.056228 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:16.056325 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:16.056450 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:16.057310 (+ 860us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60894: BlockingRecv error: recv got EOF from 127.0.0.1:60894 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260810 06:43:16.126418 5191 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:16.124629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60902 (local address 127.31.59.253:35017)
0810 06:43:16.124788 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:16.124792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:16.125160 (+ 368us) server_negotiation.cc:407] Connection header received
0810 06:43:16.125406 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:16.125410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:16.125481 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:16.125577 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:16.126204 (+ 627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60902: BlockingRecv error: recv got EOF from 127.0.0.1:60902 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260810 06:43:16.259884 5191 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:16.257831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60916 (local address 127.31.59.253:35017)
0810 06:43:16.258033 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:16.258040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:16.258399 (+ 359us) server_negotiation.cc:407] Connection header received
0810 06:43:16.258634 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:16.258641 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:16.258718 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:16.258805 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:16.259671 (+ 866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60916: BlockingRecv error: recv got EOF from 127.0.0.1:60916 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:43:16.358350 5332 catalog_manager.cc:1259] Loaded cluster ID: 2cfaebb6f7534eac938afaa497b33762
I20260810 06:43:16.358462 5332 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10: loading cluster ID for follower catalog manager: success
I20260810 06:43:16.360952 5332 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10: acquiring CA information for follower catalog manager: success
I20260810 06:43:16.362684 5332 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fc3e231ddf064f4cb65987eddae82d10: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 06:43:16.521718 5191 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:16.519918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60918 (local address 127.31.59.253:35017)
0810 06:43:16.520130 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:16.520136 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:16.520316 (+ 180us) server_negotiation.cc:407] Connection header received
0810 06:43:16.520518 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:16.520522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:16.520605 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:16.520716 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:16.521482 (+ 766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60918: BlockingRecv error: recv got EOF from 127.0.0.1:60918 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:43:16.645350 5415 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5: acquiring CA information for follower catalog manager: success
I20260810 06:43:16.647241 5415 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f0a7af5fe11e431c942d1125ec9dece5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:43:17.045181 5530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:17.039721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60926 (local address 127.31.59.253:35017)
0810 06:43:17.040170 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:17.040175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:17.040237 (+ 62us) server_negotiation.cc:407] Connection header received
0810 06:43:17.040418 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:17.040423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:17.040579 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:17.040683 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:17.041068 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:17.041749 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:17.043088 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:17.043481 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:17.043573 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:17.044108 (+ 535us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:17.044603 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:17.044719 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:17.044803 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 06:43:17.044869 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":240,"threads_started":1}
I20260810 06:43:17.059006 5540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:17.051824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58678 (local address 127.31.59.252:42429)
0810 06:43:17.054365 (+ 2541us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:17.054369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:17.054384 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:43:17.054467 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:17.054471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:17.054622 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:17.054752 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:17.055061 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:17.055744 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:17.056938 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:17.057375 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:17.057461 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:17.057924 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:17.058475 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:17.058597 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:17.058665 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 06:43:17.058727 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2417,"thread_start_us":2322,"threads_started":1}
I20260810 06:43:17.072156 5548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:17.065121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34326 (local address 127.31.59.254:33025)
0810 06:43:17.065892 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:17.065898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:17.065928 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:43:17.066024 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:17.066027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:17.066182 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:17.066355 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:17.066909 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:17.068000 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:17.069987 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:17.070637 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:17.070744 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:17.071218 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:17.071545 (+ 327us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:17.071642 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:17.071779 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 06:43:17.071860 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":546,"threads_started":1}
I20260810 06:43:17.072852 5540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:17.065092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58690 (local address 127.31.59.252:42429)
0810 06:43:17.065276 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:17.065282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:17.066951 (+ 1669us) server_negotiation.cc:407] Connection header received
0810 06:43:17.067169 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:17.067174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:17.067230 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:17.067297 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:17.067797 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:17.068744 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:17.070761 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:17.071363 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:17.071485 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:17.071917 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:17.072025 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:17.072108 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:17.072596 (+ 488us) server_negotiation.cc:299] Negotiation successful
0810 06:43:17.072671 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:43:17.073637 5530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:17.067029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60936 (local address 127.31.59.253:35017)
0810 06:43:17.067181 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:17.067187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:17.068268 (+ 1081us) server_negotiation.cc:407] Connection header received
0810 06:43:17.068420 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:17.068426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:17.068505 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:17.068598 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:17.069165 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:17.069815 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:17.071488 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:17.072058 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:17.072165 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:17.072990 (+ 825us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:17.073098 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:17.073182 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:17.073339 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 06:43:17.073412 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:43:17.077560 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 4902
I20260810 06:43:17.078469 5035 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:17.185595 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 4902
I20260810 06:43:17.212316 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 5053
I20260810 06:43:17.213395 5169 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:17.324501 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 5053
I20260810 06:43:17.344350 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 5197
I20260810 06:43:17.345486 5295 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:17.448971 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 5197
2026-08-10T06:43:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5160 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:43:17 dist-test-slave-4tdj krb5kdc[5558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:43:17 dist-test-slave-4tdj krb5kdc[5558](info): set up 2 sockets
Aug 10 06:43:17 dist-test-slave-4tdj krb5kdc[5558](info): commencing operation
krb5kdc: starting...
W20260810 06:43:18.701205 31983 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/test-user.keytab.
Aug 10 06:43:18 dist-test-slave-4tdj krb5kdc[5558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:43:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:43:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.59.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
Entry for principal kudu/127.31.59.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab.
I20260810 06:43:18.989140 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:33309
--webserver_interface=127.31.59.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:33309,127.31.59.253:36915,127.31.59.252:40155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:19.318944 5580 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:43:19.319290 5580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:19.319391 5580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:19.319512 5580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:19.326594 5580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:19.326681 5580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:19.326772 5580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:19.326848 5580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:19.326932 5580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:19.336325 5580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:33309,127.31.59.253:36915,127.31.59.252:40155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.254.keytab
--principal=kudu/127.31.59.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.254:33309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.59.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:19.337527 5580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:19.338819 5580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:19.349818 5586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:19.349915 5587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:19.350286 5590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:19.351151 5580 server_base.cc:1034] running on GCE node
I20260810 06:43:19.351869 5580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:19.353119 5580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:19.354377 5580 hybrid_clock.cc:648] HybridClock initialized: now 1786344199354299 us; error 38 us; skew 500 ppm
I20260810 06:43:19.354688 5580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:19 dist-test-slave-4tdj krb5kdc[5558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344199, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:19.357554 5580 init.cc:377] Logged in from keytab as kudu/127.31.59.254@KRBTEST.COM (short username kudu)
I20260810 06:43:19.358860 5580 webserver.cc:460] Webserver started at http://127.31.59.254:46227/ using document root <none> and password file <none>
I20260810 06:43:19.359447 5580 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:19.359529 5580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:19.359795 5580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:19.362560 5580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-0/data/instance:
uuid: "96f502192ac748fba56a0387cca3cfdf"
format_stamp: "Formatted at 2026-08-10 06:43:19 on dist-test-slave-4tdj"
server_key: "8b24e4ec735d0794d5214f62b3d0d27d"
server_key_iv: "fcb6cca441344f0657169561297a49ef"
server_key_version: "encryptionkey@0"
I20260810 06:43:19.363272 5580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-0/wal/instance:
uuid: "96f502192ac748fba56a0387cca3cfdf"
format_stamp: "Formatted at 2026-08-10 06:43:19 on dist-test-slave-4tdj"
server_key: "8b24e4ec735d0794d5214f62b3d0d27d"
server_key_iv: "fcb6cca441344f0657169561297a49ef"
server_key_version: "encryptionkey@0"
I20260810 06:43:19.367966 5580 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:43:19.371088 5602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:19.372267 5580 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:43:19.372409 5580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-0/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-0/wal
uuid: "96f502192ac748fba56a0387cca3cfdf"
format_stamp: "Formatted at 2026-08-10 06:43:19 on dist-test-slave-4tdj"
server_key: "8b24e4ec735d0794d5214f62b3d0d27d"
server_key_iv: "fcb6cca441344f0657169561297a49ef"
server_key_version: "encryptionkey@0"
I20260810 06:43:19.372592 5580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:19.389529 5580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:19.392993 5580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:19.393239 5580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:19.409737 5580 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.254:33309
I20260810 06:43:19.409757 5693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.254:33309 every 8 connection(s)
I20260810 06:43:19.410977 5580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-0/data/info.pb
I20260810 06:43:19.414608 5695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:19.415429 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 5580
I20260810 06:43:19.415601 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-0/wal/instance
I20260810 06:43:19.415992 31983 external_mini_cluster.cc:1428] Setting key a10ecec659772dbeff0b654899faf857
I20260810 06:43:19.420868 5695 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:43:19 dist-test-slave-4tdj krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344199, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:43:19.442060 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:19.423827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45908 (local address 127.31.59.254:33309)
0810 06:43:19.425447 (+ 1620us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:19.425452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:19.425504 (+ 52us) server_negotiation.cc:407] Connection header received
0810 06:43:19.427284 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:19.427289 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:19.427506 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:19.427668 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:19.429015 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:19.430536 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:19.431829 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:19.432338 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:19.432491 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:19.435510 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:19.435517 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:19.435528 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:19.435563 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:19.438029 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:19.438699 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:19.438705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:19.438707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:19.438839 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:19.439206 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:19.439210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:19.439212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:19.439647 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:19.439828 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:19.440372 (+ 544us) server_negotiation.cc:299] Negotiation successful
0810 06:43:19.440663 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1441,"thread_start_us":1198,"threads_started":1}
I20260810 06:43:19.442078 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:19.423827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:33309 (local address 127.0.0.1:45908)
0810 06:43:19.424639 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:19.424692 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:19.426927 (+ 2235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:19.427744 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:19.427751 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:19.428416 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:19.428815 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:19.428833 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:19.430682 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:19.430686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:19.431699 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:19.431711 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:19.432552 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:19.432560 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:19.432719 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:19.433269 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:19.433289 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:19.435314 (+ 2025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:19.438142 (+ 2828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:19.438153 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:19.438165 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:19.438586 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:19.438940 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:19.438945 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:19.438949 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:19.439104 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:19.439772 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:19.439779 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:19.439987 (+ 208us) client_negotiation.cc:769] Sending connection context
0810 06:43:19.440200 (+ 213us) client_negotiation.cc:240] Negotiation successful
0810 06:43:19.440788 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":179,"thread_start_us":317,"threads_started":1}
WARNING: no policy specified for kudu/127.31.59.253@KRBTEST.COM; defaulting to no policy
I20260810 06:43:19.444926 5695 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:43:19.445900 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:19.445510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36915 (local address 127.0.0.1:60212)
0810 06:43:19.445675 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:19.445796 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 06:43:19.446098 5603 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36915: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.253@KRBTEST.COM" created.
I20260810 06:43:19.447595 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:19.447191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36915 (local address 127.0.0.1:60224)
0810 06:43:19.447335 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:19.447415 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 06:43:19.447788 5695 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.253:36915: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36915: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.253.keytab.
Entry for principal kudu/127.31.59.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.253.keytab.
I20260810 06:43:19.479199 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.253:36915
--webserver_interface=127.31.59.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.253.keytab
--principal=kudu/127.31.59.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:33309,127.31.59.253:36915,127.31.59.252:40155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:43:19.497107 5695 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } attempt: 1
I20260810 06:43:19.498409 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:19.497870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36915 (local address 127.0.0.1:60228)
0810 06:43:19.498073 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:19.498210 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 06:43:19.499552 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:19.499146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36915 (local address 127.0.0.1:60240)
0810 06:43:19.499303 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:19.499377 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 06:43:19.499770 5695 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.253:36915: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36915: connect: Connection refused (error 111)
I20260810 06:43:19.568029 5695 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } attempt: 2
I20260810 06:43:19.569540 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:19.568847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36915 (local address 127.0.0.1:60252)
0810 06:43:19.569165 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:19.569331 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 06:43:19.570966 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:19.570451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36915 (local address 127.0.0.1:60266)
0810 06:43:19.570657 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:19.570810 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 06:43:19.571192 5695 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.253:36915: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36915: connect: Connection refused (error 111)
I20260810 06:43:19.662461 5695 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } attempt: 3
I20260810 06:43:19.663542 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:19.663107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36915 (local address 127.0.0.1:60278)
0810 06:43:19.663313 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:19.663408 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 06:43:19.664551 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:19.664201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36915 (local address 127.0.0.1:60294)
0810 06:43:19.664345 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:19.664412 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 06:43:19.664721 5695 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.253:36915: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36915: connect: Connection refused (error 111)
I20260810 06:43:19.807936 5695 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } attempt: 4
I20260810 06:43:19.809075 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:19.808560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36915 (local address 127.0.0.1:60296)
0810 06:43:19.808840 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:19.808923 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260810 06:43:19.810072 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:19.809732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36915 (local address 127.0.0.1:60298)
0810 06:43:19.809870 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:19.809945 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 06:43:19.810261 5695 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.253:36915: Network error: Client connection negotiation failed: client connection to 127.31.59.253:36915: connect: Connection refused (error 111)
W20260810 06:43:19.811247 5707 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:43:19.811576 5707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:19.811687 5707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:19.811820 5707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:19.819087 5707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:19.819182 5707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:19.819288 5707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:19.819361 5707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:19.819455 5707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:19.829124 5707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:33309,127.31.59.253:36915,127.31.59.252:40155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.253.keytab
--principal=kudu/127.31.59.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.253:36915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.59.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:19.830438 5707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:19.831765 5707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:19.842898 5716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:19.842929 5715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:19.843081 5718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:19.843815 5707 server_base.cc:1034] running on GCE node
I20260810 06:43:19.844488 5707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:19.845707 5707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:19.846908 5707 hybrid_clock.cc:648] HybridClock initialized: now 1786344199846850 us; error 38 us; skew 500 ppm
I20260810 06:43:19.847203 5707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:19 dist-test-slave-4tdj krb5kdc[5558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344199, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:19.850068 5707 init.cc:377] Logged in from keytab as kudu/127.31.59.253@KRBTEST.COM (short username kudu)
I20260810 06:43:19.851414 5707 webserver.cc:460] Webserver started at http://127.31.59.253:38941/ using document root <none> and password file <none>
I20260810 06:43:19.852016 5707 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:19.852123 5707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:19.852403 5707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:19.855182 5707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-1/data/instance:
uuid: "e629c89fdc45451488bdba0cf400c20d"
format_stamp: "Formatted at 2026-08-10 06:43:19 on dist-test-slave-4tdj"
server_key: "52c8243c825fead1066051b11f8c1e52"
server_key_iv: "1202c5737f9c289f83a75bab424791b8"
server_key_version: "encryptionkey@0"
I20260810 06:43:19.855909 5707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-1/wal/instance:
uuid: "e629c89fdc45451488bdba0cf400c20d"
format_stamp: "Formatted at 2026-08-10 06:43:19 on dist-test-slave-4tdj"
server_key: "52c8243c825fead1066051b11f8c1e52"
server_key_iv: "1202c5737f9c289f83a75bab424791b8"
server_key_version: "encryptionkey@0"
I20260810 06:43:19.860627 5707 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:43:19.863866 5739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:19.865087 5707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:43:19.865245 5707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-1/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-1/wal
uuid: "e629c89fdc45451488bdba0cf400c20d"
format_stamp: "Formatted at 2026-08-10 06:43:19 on dist-test-slave-4tdj"
server_key: "52c8243c825fead1066051b11f8c1e52"
server_key_iv: "1202c5737f9c289f83a75bab424791b8"
server_key_version: "encryptionkey@0"
I20260810 06:43:19.865435 5707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:19.883227 5707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:19.887190 5707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:19.887432 5707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:19.903812 5707 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.253:36915
I20260810 06:43:19.903831 5820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.253:36915 every 8 connection(s)
I20260810 06:43:19.905067 5707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-1/data/info.pb
I20260810 06:43:19.906986 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 5707
I20260810 06:43:19.907181 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-1/wal/instance
I20260810 06:43:19.907727 31983 external_mini_cluster.cc:1428] Setting key 78e20e16a875c0fb2c4a7b9b35a63478
I20260810 06:43:19.909268 5821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:19.917389 5821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:43:19 dist-test-slave-4tdj krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344199, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.253@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:43:19.937896 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:19.920416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45922 (local address 127.31.59.254:33309)
0810 06:43:19.920630 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:19.920636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:19.921842 (+ 1206us) server_negotiation.cc:407] Connection header received
0810 06:43:19.923735 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:19.923741 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:19.923826 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:19.923935 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:19.925332 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:19.926265 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:19.927726 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:19.928331 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:19.928444 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:19.931822 (+ 3378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:19.931828 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:19.931832 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:19.931869 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:19.935002 (+ 3133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:19.935667 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:19.935672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:19.935676 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:19.935824 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:19.936183 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:19.936188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:19.936192 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:19.936548 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:19.936704 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:19.937328 (+ 624us) server_negotiation.cc:299] Negotiation successful
0810 06:43:19.937562 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:43:19.938606 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:19.920894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:33309 (local address 127.0.0.1:45922)
0810 06:43:19.921680 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:19.921725 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:19.923474 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:19.924113 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:19.924120 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:19.924618 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:19.925147 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:19.925164 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:19.926413 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:19.926417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:19.927558 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:19.927571 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:19.928540 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:19.928546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:19.928753 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:19.929361 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:19.929384 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:19.931602 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:19.935123 (+ 3521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:19.935130 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:19.935140 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:19.935550 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:19.935918 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:19.935921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:19.935923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:19.936084 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:19.936675 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:19.936680 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:19.936959 (+ 279us) client_negotiation.cc:769] Sending connection context
0810 06:43:19.937191 (+ 232us) client_negotiation.cc:240] Negotiation successful
0810 06:43:19.937537 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":337,"threads_started":1}
WARNING: no policy specified for kudu/127.31.59.252@KRBTEST.COM; defaulting to no policy
I20260810 06:43:19.940222 5821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.59.252@KRBTEST.COM" created.
Aug 10 06:43:19 dist-test-slave-4tdj krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344199, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.253@KRBTEST.COM for kudu/127.31.59.253@KRBTEST.COM
I20260810 06:43:19.955215 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:19.940869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36915 (local address 127.0.0.1:60310)
0810 06:43:19.941066 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:19.941091 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:19.941174 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:19.942450 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:19.942453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:19.942804 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:19.942939 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:19.942947 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:19.944818 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:19.944822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:19.945661 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:19.945672 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:19.946640 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:19.946644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:19.946794 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:19.947299 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:19.947315 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:19.949174 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:19.952728 (+ 3554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:19.952732 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:19.952735 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:19.953106 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:19.953504 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:19.953508 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:19.953511 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:19.953658 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:19.954268 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:19.954273 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:19.954438 (+ 165us) client_negotiation.cc:769] Sending connection context
0810 06:43:19.954584 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 06:43:19.954847 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":1}
I20260810 06:43:19.955235 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:19.941153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60310 (local address 127.31.59.253:36915)
0810 06:43:19.941927 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:19.941934 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:19.941967 (+ 33us) server_negotiation.cc:407] Connection header received
0810 06:43:19.942038 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:19.942045 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:19.942263 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:19.942449 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:19.943112 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:19.944676 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:19.945789 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:19.946515 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:19.946632 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:19.949371 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:19.949383 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:19.949397 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:19.949429 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:19.952603 (+ 3174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:19.953224 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:19.953232 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:19.953235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:19.953367 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:19.953758 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:19.953763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:19.953766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:19.954145 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:19.954349 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:19.954702 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 06:43:19.954945 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":99,"server-negotiator.queue_time_us":632,"thread_start_us":392,"threads_started":1}
I20260810 06:43:19.957537 5821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:43:19.958604 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:19.958097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:51894)
0810 06:43:19.958323 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:19.958441 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 06:43:19.958851 5742 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
I20260810 06:43:19.960306 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:19.959945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:51896)
0810 06:43:19.960083 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:19.960192 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 06:43:19.960552 5821 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:40155: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.59.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.252.keytab.
Entry for principal kudu/127.31.59.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.252.keytab.
I20260810 06:43:19.976917 31983 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
/tmp/dist-test-taskq68xXZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.252:40155
--webserver_interface=127.31.59.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.252.keytab
--principal=kudu/127.31.59.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.59.212:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.59.254:33309,127.31.59.253:36915,127.31.59.252:40155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:43:20.009853 5821 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } attempt: 1
I20260810 06:43:20.011049 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.010500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:51910)
0810 06:43:20.010732 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.010861 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 06:43:20.012084 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.011694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:51918)
0810 06:43:20.011846 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.011939 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 06:43:20.012291 5821 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:40155: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
I20260810 06:43:20.080524 5821 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } attempt: 2
I20260810 06:43:20.081712 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.081176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:51928)
0810 06:43:20.081395 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.081513 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 06:43:20.082747 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.082374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:51940)
0810 06:43:20.082517 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.082605 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 06:43:20.082985 5821 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:40155: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
I20260810 06:43:20.109505 5695 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } attempt: 5
Aug 10 06:43:20 dist-test-slave-4tdj krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344199, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for kudu/127.31.59.253@KRBTEST.COM
I20260810 06:43:20.122429 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.110246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60314 (local address 127.31.59.253:36915)
0810 06:43:20.110440 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:20.110445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:20.110556 (+ 111us) server_negotiation.cc:407] Connection header received
0810 06:43:20.110755 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:20.110759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:20.110824 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:20.110910 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:20.111705 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.112392 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.113560 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.114011 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.114128 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:20.117401 (+ 3273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:20.117406 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:20.117409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:20.117442 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:20.120019 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.120634 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.120638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.120641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.120757 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.121104 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.121109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.121112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.121380 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:20.121519 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:20.121889 (+ 370us) server_negotiation.cc:299] Negotiation successful
0810 06:43:20.122101 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 06:43:20.122429 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.110224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36915 (local address 127.0.0.1:60314)
0810 06:43:20.110439 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.110458 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:20.110619 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:20.110957 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:20.110961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:20.111364 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:20.111555 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:20.111565 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.112527 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.112533 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:20.113396 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:20.113409 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.114162 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.114170 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:20.114402 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:20.114998 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:20.115015 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:20.117220 (+ 2205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:20.120144 (+ 2924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:20.120150 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:20.120155 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:20.120522 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:20.120860 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:20.120865 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:20.120869 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:20.121006 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:20.121492 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:20.121497 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:20.121643 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 06:43:20.121765 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 06:43:20.122040 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260810 06:43:20.123649 5695 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:43:20.124730 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.124373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:51954)
0810 06:43:20.124530 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.124605 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 06:43:20.125690 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.125393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:51970)
0810 06:43:20.125516 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.125598 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 06:43:20.125898 5695 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:40155: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
I20260810 06:43:20.174221 5821 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } attempt: 3
I20260810 06:43:20.175516 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.174960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:51980)
0810 06:43:20.175186 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.175321 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 06:43:20.176664 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.176198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:51982)
0810 06:43:20.176422 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.176518 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260810 06:43:20.176934 5821 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:40155: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
I20260810 06:43:20.177098 5695 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } attempt: 1
I20260810 06:43:20.177960 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.177571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:51996)
0810 06:43:20.177731 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.177813 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 06:43:20.178835 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.178528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:52000)
0810 06:43:20.178666 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.178739 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 06:43:20.179016 5695 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:40155: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
I20260810 06:43:20.247225 5695 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } attempt: 2
I20260810 06:43:20.248297 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.247837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:52014)
0810 06:43:20.248047 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.248135 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 06:43:20.249208 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.248891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:52020)
0810 06:43:20.249028 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.249109 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 06:43:20.249408 5695 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:40155: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
W20260810 06:43:20.304221 5838 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:43:20.304540 5838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:20.304641 5838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:20.304755 5838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:20.311702 5838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:20.311789 5838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:20.311872 5838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:20.311934 5838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:20.312016 5838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:20.320153 5821 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } attempt: 4
I20260810 06:43:20.321380 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.320830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:52022)
0810 06:43:20.321083 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.321192 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 06:43:20.322021 5838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.59.212:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.59.254:33309,127.31.59.253:36915,127.31.59.252:40155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/krb5kdc/kudu_127.31.59.252.keytab
--principal=kudu/127.31.59.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.59.252:40155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.59.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:20.322376 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.321990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:52024)
0810 06:43:20.322141 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.322226 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 06:43:20.322585 5821 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:40155: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
I20260810 06:43:20.323434 5838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:20.324672 5838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:20.335806 5848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:20.335769 5850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:20.336175 5838 server_base.cc:1034] running on GCE node
W20260810 06:43:20.335834 5847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:20.337169 5838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:20.338476 5838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:20.339753 5838 hybrid_clock.cc:648] HybridClock initialized: now 1786344200339668 us; error 50 us; skew 500 ppm
I20260810 06:43:20.340049 5838 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:20 dist-test-slave-4tdj krb5kdc[5558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344200, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:20.343170 5838 init.cc:377] Logged in from keytab as kudu/127.31.59.252@KRBTEST.COM (short username kudu)
I20260810 06:43:20.344421 5838 webserver.cc:460] Webserver started at http://127.31.59.252:40095/ using document root <none> and password file <none>
I20260810 06:43:20.345052 5838 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:20.345135 5838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:20.345413 5838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:20.348281 5838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-2/data/instance:
uuid: "7a3d7b807f434f3e872572eee52ed96b"
format_stamp: "Formatted at 2026-08-10 06:43:20 on dist-test-slave-4tdj"
server_key: "fd023e479043ad72e1645619c0e09a48"
server_key_iv: "8a1964127496e314bcd8a8ead01e99ef"
server_key_version: "encryptionkey@0"
I20260810 06:43:20.349001 5838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-2/wal/instance:
uuid: "7a3d7b807f434f3e872572eee52ed96b"
format_stamp: "Formatted at 2026-08-10 06:43:20 on dist-test-slave-4tdj"
server_key: "fd023e479043ad72e1645619c0e09a48"
server_key_iv: "8a1964127496e314bcd8a8ead01e99ef"
server_key_version: "encryptionkey@0"
I20260810 06:43:20.353890 5838 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260810 06:43:20.355592 5695 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } attempt: 3
I20260810 06:43:20.356671 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.356151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:52028)
0810 06:43:20.356364 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.356456 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 06:43:20.357362 5864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:20.357645 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.357257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:52036)
0810 06:43:20.357452 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.357542 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260810 06:43:20.357836 5695 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.59.252:40155: Network error: Client connection negotiation failed: client connection to 127.31.59.252:40155: connect: Connection refused (error 111)
I20260810 06:43:20.358741 5838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 06:43:20.358906 5838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-2/data,/tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-2/wal
uuid: "7a3d7b807f434f3e872572eee52ed96b"
format_stamp: "Formatted at 2026-08-10 06:43:20 on dist-test-slave-4tdj"
server_key: "fd023e479043ad72e1645619c0e09a48"
server_key_iv: "8a1964127496e314bcd8a8ead01e99ef"
server_key_version: "encryptionkey@0"
I20260810 06:43:20.359107 5838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:20.386361 5838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:20.390611 5838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:20.390866 5838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:20.407507 5838 rpc_server.cc:307] RPC server started. Bound to: 127.31.59.252:40155
I20260810 06:43:20.407542 5950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.59.252:40155 every 8 connection(s)
I20260810 06:43:20.408798 5838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-2/data/info.pb
I20260810 06:43:20.412592 5952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:20.413555 31983 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu as pid 5838
I20260810 06:43:20.413744 31983 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq68xXZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343831713822-31983-0/minicluster-data/master-2/wal/instance
I20260810 06:43:20.414314 31983 external_mini_cluster.cc:1428] Setting key d728146dba698758cb4e7c33eacab062
I20260810 06:43:20.422108 5952 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:43:20 dist-test-slave-4tdj krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.252@KRBTEST.COM
Aug 10 06:43:20 dist-test-slave-4tdj krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344200, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.252@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:43:20.437585 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.416195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52042 (local address 127.31.59.252:40155)
0810 06:43:20.418666 (+ 2471us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:20.418677 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:20.418718 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:43:20.420237 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:20.420262 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:20.420636 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:20.420950 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:20.422383 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.423645 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.425395 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.426400 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.426613 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:20.430322 (+ 3709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:20.430334 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:20.430348 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:20.430391 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:20.433590 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.434331 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.434340 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.434348 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.434495 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.434887 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.434892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.434895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.435382 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:20.435591 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:20.436057 (+ 466us) server_negotiation.cc:299] Negotiation successful
0810 06:43:20.436377 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":2242,"thread_start_us":1997,"threads_started":1}
I20260810 06:43:20.438881 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.424442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45930 (local address 127.31.59.254:33309)
0810 06:43:20.424623 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:20.424627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:20.425285 (+ 658us) server_negotiation.cc:407] Connection header received
0810 06:43:20.425534 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:20.425538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:20.425603 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:20.425679 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:20.427091 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.427797 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.429066 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.429500 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.429612 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:20.433288 (+ 3676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:20.433293 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:20.433296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:20.433327 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:20.436152 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.436933 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.436936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.436939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.437077 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.437455 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.437457 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.437459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.437720 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:20.437860 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:20.438339 (+ 479us) server_negotiation.cc:299] Negotiation successful
0810 06:43:20.438536 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:43:20.438946 5958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.424509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.254:33309 (local address 127.0.0.1:45930)
0810 06:43:20.425133 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.425176 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:20.425348 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:20.425752 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:20.425758 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:20.426599 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:20.426945 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:20.426955 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.427945 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.427950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:20.428924 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:20.428936 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.429713 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.429721 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:20.429862 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:20.430515 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:20.430537 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:20.433103 (+ 2566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:20.436276 (+ 3173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:20.436286 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:20.436293 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:20.436818 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:20.437181 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:20.437186 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:20.437189 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:20.437368 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:20.437854 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:20.437861 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:20.438021 (+ 160us) client_negotiation.cc:769] Sending connection context
0810 06:43:20.438205 (+ 184us) client_negotiation.cc:240] Negotiation successful
0810 06:43:20.438536 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":2,"thread_start_us":306,"threads_started":1}
I20260810 06:43:20.440251 5952 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:43:20 dist-test-slave-4tdj krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344200, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.252@KRBTEST.COM for kudu/127.31.59.253@KRBTEST.COM
Aug 10 06:43:20 dist-test-slave-4tdj krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.254@KRBTEST.COM
I20260810 06:43:20.454499 5958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.440908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.253:36915 (local address 127.0.0.1:60320)
0810 06:43:20.441148 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.441165 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:20.441270 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:20.441595 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:20.441600 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:20.441997 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:20.442157 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:20.442168 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.443581 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.443586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:20.444960 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:20.444975 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.445887 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.445893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:20.446095 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:20.446688 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:20.446710 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:20.449001 (+ 2291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:20.452336 (+ 3335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:20.452340 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:20.452342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:20.452686 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:20.453053 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:20.453056 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:20.453058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:20.453196 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:20.453678 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:20.453683 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:20.453824 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 06:43:20.453945 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 06:43:20.454212 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260810 06:43:20.454589 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.440942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60320 (local address 127.31.59.253:36915)
0810 06:43:20.441105 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:20.441109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:20.441260 (+ 151us) server_negotiation.cc:407] Connection header received
0810 06:43:20.441376 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:20.441380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:20.441453 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:20.441542 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:20.442332 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.443417 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.445132 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.445764 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.445869 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:20.449170 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:20.449175 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:20.449178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:20.449208 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:20.452195 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.452797 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.452803 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.452807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.452960 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.453300 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.453303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.453305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.453563 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:20.453668 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:20.454036 (+ 368us) server_negotiation.cc:299] Negotiation successful
0810 06:43:20.454188 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:43:20.455453 5952 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:43:20.455790 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.440312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45934 (local address 127.31.59.254:33309)
0810 06:43:20.440458 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:20.440463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:20.440591 (+ 128us) server_negotiation.cc:407] Connection header received
0810 06:43:20.440741 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:20.440746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:20.440808 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:20.440894 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:20.442330 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.443433 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.445132 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.445737 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.445860 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:20.450207 (+ 4347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:20.450214 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:20.450218 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:20.450255 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:20.453134 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.453726 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.453731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.453735 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.453856 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.454228 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.454232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.454235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.454571 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:20.454840 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:20.455261 (+ 421us) server_negotiation.cc:299] Negotiation successful
0810 06:43:20.455472 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 06:43:20 dist-test-slave-4tdj krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344200, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.252@KRBTEST.COM for kudu/127.31.59.252@KRBTEST.COM
Aug 10 06:43:20 dist-test-slave-4tdj krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.59.253@KRBTEST.COM
I20260810 06:43:20.468182 5958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.456194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:52048)
0810 06:43:20.456342 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.456356 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:20.456427 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:20.456760 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:20.456763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:20.457064 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:20.457182 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:20.457189 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.458436 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.458440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:20.459440 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:20.459454 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.460148 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.460152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:20.460319 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:20.460815 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:20.460831 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:20.463083 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:20.465728 (+ 2645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:20.465733 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:20.465736 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:20.466186 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:20.466611 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:20.466614 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:20.466616 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:20.466740 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:20.467346 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:20.467350 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:20.467494 (+ 144us) client_negotiation.cc:769] Sending connection context
0810 06:43:20.467614 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 06:43:20.467899 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 06:43:20.468456 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.456393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52048 (local address 127.31.59.252:40155)
0810 06:43:20.456535 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:20.456540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:20.456555 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:43:20.456588 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:20.456591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:20.456635 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:20.456723 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:20.457306 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.458314 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.459603 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.460008 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.460135 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:20.463256 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:20.463261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:20.463264 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:20.463291 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:20.465597 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.466345 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.466350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.466353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.466510 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.466849 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.466853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.466856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.467211 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:20.467358 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:20.467724 (+ 366us) server_negotiation.cc:299] Negotiation successful
0810 06:43:20.468088 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":118,"server-negotiator.queue_time_us":48}
I20260810 06:43:20.472101 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.457369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60326 (local address 127.31.59.253:36915)
0810 06:43:20.457508 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:20.457511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:20.457528 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:43:20.457679 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:20.457682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:20.457728 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:20.457794 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:20.459063 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.459904 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.461224 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.461819 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.461949 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:20.466136 (+ 4187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:20.466140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:20.466143 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:20.466173 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:20.469612 (+ 3439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.470270 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.470275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.470278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.470467 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.470787 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.470792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.470795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.471126 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:20.471391 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:20.471542 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 06:43:20.471754 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:43:20.474072 5952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b: Bootstrap starting.
I20260810 06:43:20.476717 31983 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:43:20.477849 5952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:20.479157 5952 log.cc:824] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:20.481863 5952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b: No bootstrap required, opened a new log
I20260810 06:43:20.479655 5643 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "96f502192ac748fba56a0387cca3cfdf"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45934
I20260810 06:43:20.488668 5952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } } peers { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } } peers { permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } }
I20260810 06:43:20.488966 5952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a3d7b807f434f3e872572eee52ed96b, State: Initialized, Role: FOLLOWER
I20260810 06:43:20.489359 5952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } } peers { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } } peers { permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } }
I20260810 06:43:20.490268 5961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } } peers { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } } peers { permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } } }
I20260810 06:43:20.490664 5961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:43:20.491274 5952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b [sys.catalog]: configured and running, proceeding with master startup.
W20260810 06:43:20.500828 5975 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:43:20.500931 5975 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:43:20.499339 5838 master_runner.cc:186] Error getting master registration for 127.31.59.254:33309: OK, instance_id { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" instance_seqno: 1786344199404921 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:43:20.510035 5838 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:43:20.535046 5695 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } attempt: 4
Aug 10 06:43:20 dist-test-slave-4tdj krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344199, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.254@KRBTEST.COM for kudu/127.31.59.252@KRBTEST.COM
I20260810 06:43:20.551177 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.535832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:52064)
0810 06:43:20.536045 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.536066 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:20.536206 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:20.536554 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:20.536558 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:20.536972 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:20.537145 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:20.537155 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.538672 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.538678 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:20.539616 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:20.539629 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.540620 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.540627 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:20.540817 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:20.541449 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:20.541473 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:20.544235 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:20.548355 (+ 4120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:20.548360 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:20.548365 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:20.548880 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:20.549263 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:20.549268 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:20.549272 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:20.549421 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:20.550125 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:20.550130 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:20.550341 (+ 211us) client_negotiation.cc:769] Sending connection context
0810 06:43:20.550484 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 06:43:20.550769 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 06:43:20.551208 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.535901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52064 (local address 127.31.59.252:40155)
0810 06:43:20.536071 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:20.536077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:20.536146 (+ 69us) server_negotiation.cc:407] Connection header received
0810 06:43:20.536337 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:20.536342 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:20.536423 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:20.536524 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:20.537294 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.538520 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.539762 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.540465 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.540590 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:20.544434 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:20.544441 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:20.544446 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:20.544489 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:20.548212 (+ 3723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.549005 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.549010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.549013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.549159 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.549530 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.549535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.549538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.550010 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:20.550155 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:20.550600 (+ 445us) server_negotiation.cc:299] Negotiation successful
0810 06:43:20.550823 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:43:20.557284 5695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf: Bootstrap starting.
I20260810 06:43:20.561266 5695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:20.562650 5695 log.cc:824] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:20.565655 5695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf: No bootstrap required, opened a new log
I20260810 06:43:20.572194 5695 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } } peers { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } } peers { permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } }
I20260810 06:43:20.572614 5695 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f502192ac748fba56a0387cca3cfdf, State: Initialized, Role: FOLLOWER
I20260810 06:43:20.573133 5695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } } peers { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } } peers { permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } }
I20260810 06:43:20.580070 5982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } } peers { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } } peers { permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } } }
I20260810 06:43:20.580430 5982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:43:20.581812 5695 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:43:20.581924 5643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 06:43:20.582139 5643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:20.585676 5643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } } peers { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } } peers { permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } }
I20260810 06:43:20.588651 5913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "96f502192ac748fba56a0387cca3cfdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7a3d7b807f434f3e872572eee52ed96b"
I20260810 06:43:20.589164 5913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:20.592697 5913 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96f502192ac748fba56a0387cca3cfdf in term 1.
I20260810 06:43:20.593088 5643 leader_election.cc:288] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [CANDIDATE]: Term 1 election: Requested vote from peers e629c89fdc45451488bdba0cf400c20d (127.31.59.253:36915), 7a3d7b807f434f3e872572eee52ed96b (127.31.59.252:40155)
I20260810 06:43:20.593369 5606 leader_election.cc:302] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a3d7b807f434f3e872572eee52ed96b, 96f502192ac748fba56a0387cca3cfdf; no voters:
I20260810 06:43:20.593963 5982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:20.607081 5982 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [term 1 LEADER]: Becoming Leader. State: Replica: 96f502192ac748fba56a0387cca3cfdf, State: Running, Role: LEADER
I20260810 06:43:20.607748 5982 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } } peers { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } } peers { permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } }
I20260810 06:43:20.622677 5991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96f502192ac748fba56a0387cca3cfdf. Latest consensus state: current_term: 1 leader_uuid: "96f502192ac748fba56a0387cca3cfdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } } peers { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } } peers { permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } } }
I20260810 06:43:20.623059 5991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:20.623377 5821 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } attempt: 5
I20260810 06:43:20.624330 6012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260810 06:43:20.642943 5878 tablet.cc:2369] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:43:20.645790 5611 tablet.cc:2369] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:43:20.654472 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.611642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60342 (local address 127.31.59.253:36915)
0810 06:43:20.611987 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:20.611992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:20.622820 (+ 10828us) server_negotiation.cc:407] Connection header received
0810 06:43:20.623085 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:20.623091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:20.623177 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:20.623247 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:20.638400 (+ 15153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.639487 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.641408 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.642005 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.642099 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:20.645616 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:20.645623 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:20.645628 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:20.645670 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:20.649097 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.649969 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.649976 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.649980 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.650322 (+ 342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.650791 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.650818 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.650821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.651367 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:20.651761 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:20.652529 (+ 768us) server_negotiation.cc:299] Negotiation successful
0810 06:43:20.654070 (+ 1541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1266,"server-negotiator.queue_time_us":219}
Aug 10 06:43:20 dist-test-slave-4tdj krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344199, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.59.253@KRBTEST.COM for kudu/127.31.59.252@KRBTEST.COM
I20260810 06:43:20.673384 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.609831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45950 (local address 127.31.59.254:33309)
0810 06:43:20.610044 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:20.610051 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:20.618487 (+ 8436us) server_negotiation.cc:407] Connection header received
0810 06:43:20.619245 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:20.619251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:20.619334 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:20.619415 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:20.638058 (+ 18643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.639229 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.647855 (+ 8626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.648538 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.648645 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:20.666011 (+ 17366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:20.666018 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:20.666024 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:20.666070 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:20.669960 (+ 3890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.670895 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.670901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.670905 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.671085 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.671524 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.671529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.671533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.671894 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:20.672177 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:20.672720 (+ 543us) server_negotiation.cc:299] Negotiation successful
0810 06:43:20.672981 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:43:20.676604 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.623926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.59.252:40155 (local address 127.0.0.1:52092)
0810 06:43:20.624290 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:20.624308 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:20.624408 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:20.642903 (+ 18495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:20.642908 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:20.643411 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:20.643601 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:20.643614 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.652323 (+ 8709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.652329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:20.653722 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:20.653738 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.659609 (+ 5871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.659616 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:20.659876 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:20.660462 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:20.660485 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:20.663408 (+ 2923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:20.672636 (+ 9228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:20.672642 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:20.672646 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:20.673178 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:20.674128 (+ 950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:20.674148 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:20.674152 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:20.674329 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:20.675675 (+ 1346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:20.675691 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:20.675840 (+ 149us) client_negotiation.cc:769] Sending connection context
0810 06:43:20.675967 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 06:43:20.676244 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251}
I20260810 06:43:20.678828 6019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.624603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52092 (local address 127.31.59.252:40155)
0810 06:43:20.642238 (+ 17635us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:20.642244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:20.642265 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:43:20.642348 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:20.642353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:20.642426 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:20.645924 (+ 3498us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:20.646184 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.647376 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.656343 (+ 8967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.657018 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.661929 (+ 4911us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:20.663812 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:20.663820 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:20.663826 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:20.663919 (+ 93us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:20.672339 (+ 8420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.673459 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.673465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.673469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.673891 (+ 422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.674583 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.674589 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.674592 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.675432 (+ 840us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:20.676936 (+ 1504us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:20.677267 (+ 331us) server_negotiation.cc:299] Negotiation successful
0810 06:43:20.677803 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17511,"thread_start_us":18101,"threads_started":1}
I20260810 06:43:20.679270 6012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:43:20.685892 5821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d: Bootstrap starting.
I20260810 06:43:20.690189 5821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:20.694958 5821 log.cc:824] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:20.696277 5913 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b [term 1 FOLLOWER]: Refusing update from remote peer 96f502192ac748fba56a0387cca3cfdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:20.697330 5991 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:43:20.698264 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:20.611457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52080 (local address 127.31.59.252:40155)
0810 06:43:20.611837 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:20.611843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:20.641803 (+ 29960us) server_negotiation.cc:407] Connection header received
0810 06:43:20.643958 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:20.643964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:20.644084 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:20.644156 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:20.649581 (+ 5425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.650571 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.662169 (+ 11598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:20.662905 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:20.663025 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:20.675719 (+ 12694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:20.675729 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:20.675734 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:20.675775 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:20.679861 (+ 4086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.694396 (+ 14535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.694403 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.694406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.694618 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:20.695682 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:20.695688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:20.695692 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:20.696102 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:20.696360 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:20.697509 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0810 06:43:20.697894 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263}
I20260810 06:43:20.712960 5821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d: No bootstrap required, opened a new log
I20260810 06:43:20.717065 5961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96f502192ac748fba56a0387cca3cfdf. Latest consensus state: current_term: 1 leader_uuid: "96f502192ac748fba56a0387cca3cfdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } } peers { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } } peers { permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } } }
I20260810 06:43:20.717417 5961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:43:20.724208 5821 raft_consensus.cc:348] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } } peers { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } } peers { permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } }
I20260810 06:43:20.724692 5821 raft_consensus.cc:729] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e629c89fdc45451488bdba0cf400c20d, State: Initialized, Role: FOLLOWER
I20260810 06:43:20.697165 5580 master_runner.cc:186] Error getting master registration for 127.31.59.253:36915: OK, instance_id { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" instance_seqno: 1786344199899083 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:43:20.725471 5580 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:43:20.725261 5821 consensus_queue.cc:260] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } } peers { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } } peers { permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } }
I20260810 06:43:20.725639 6025 mvcc.cc:205] Tried to move back new op lower bound from 7316865846037278720 to 7316865845694820352. Current Snapshot: MvccSnapshot[applied={T|T < 7316865846037278720}]
I20260810 06:43:20.731680 5991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "96f502192ac748fba56a0387cca3cfdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } } peers { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } } peers { permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } } }
I20260810 06:43:20.731959 5991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:20.732632 6032 sys_catalog.cc:455] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } } peers { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } } peers { permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } } }
I20260810 06:43:20.734875 6032 sys_catalog.cc:458] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:43:20.734748 5821 sys_catalog.cc:564] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:43:20.734799 5775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "96f502192ac748fba56a0387cca3cfdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e629c89fdc45451488bdba0cf400c20d"
I20260810 06:43:20.735321 5775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:20.738633 5775 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96f502192ac748fba56a0387cca3cfdf in term 1.
I20260810 06:43:20.739321 5774 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d [term 1 FOLLOWER]: Refusing update from remote peer 96f502192ac748fba56a0387cca3cfdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:20.741168 5961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "96f502192ac748fba56a0387cca3cfdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } } peers { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } } peers { permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } } }
I20260810 06:43:20.741242 5982 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:43:20.741427 5961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:43:20.756562 6012 catalog_manager.cc:1347] Generated new cluster ID: fcde5cef235e40ceb910bb6dec86a15a
I20260810 06:43:20.756676 6012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:43:20.780861 6012 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:43:20.782974 6012 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:43:20.795186 6041 sys_catalog.cc:455] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "96f502192ac748fba56a0387cca3cfdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } } peers { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } } peers { permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } } }
I20260810 06:43:20.795619 5991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "96f502192ac748fba56a0387cca3cfdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } } peers { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } } peers { permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } } }
I20260810 06:43:20.795871 5991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:20.795889 6041 sys_catalog.cc:458] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:43:20.796202 6032 sys_catalog.cc:455] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96f502192ac748fba56a0387cca3cfdf. Latest consensus state: current_term: 1 leader_uuid: "96f502192ac748fba56a0387cca3cfdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f502192ac748fba56a0387cca3cfdf" member_type: VOTER last_known_addr { host: "127.31.59.254" port: 33309 } } peers { permanent_uuid: "e629c89fdc45451488bdba0cf400c20d" member_type: VOTER last_known_addr { host: "127.31.59.253" port: 36915 } } peers { permanent_uuid: "7a3d7b807f434f3e872572eee52ed96b" member_type: VOTER last_known_addr { host: "127.31.59.252" port: 40155 } } }
I20260810 06:43:20.796451 6032 sys_catalog.cc:458] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:43:20.821655 6012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96f502192ac748fba56a0387cca3cfdf: Generated new TSK 0
I20260810 06:43:20.823096 6012 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:43:20.845638 6058 catalog_manager.cc:1259] Loaded cluster ID: fcde5cef235e40ceb910bb6dec86a15a
I20260810 06:43:20.845839 6058 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d: loading cluster ID for follower catalog manager: success
I20260810 06:43:20.849640 6058 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d: acquiring CA information for follower catalog manager: success
I20260810 06:43:20.852137 6058 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e629c89fdc45451488bdba0cf400c20d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 06:43:20.964932 5701 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:20.951945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45956 (local address 127.31.59.254:33309)
0810 06:43:20.952164 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:20.952170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:20.962840 (+ 10670us) server_negotiation.cc:407] Connection header received
0810 06:43:20.963183 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:20.963189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:20.963277 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:20.963352 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:20.964655 (+ 1303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45956: BlockingRecv error: recv got EOF from 127.0.0.1:45956 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260810 06:43:20.967396 5834 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:20.956172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60354 (local address 127.31.59.253:36915)
0810 06:43:20.956384 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:20.956390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:20.965797 (+ 9407us) server_negotiation.cc:407] Connection header received
0810 06:43:20.966023 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:20.966029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:20.966120 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:20.966196 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:20.967180 (+ 984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60354: BlockingRecv error: recv got EOF from 127.0.0.1:60354 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260810 06:43:20.970414 5957 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:20.963147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52104 (local address 127.31.59.252:40155)
0810 06:43:20.963340 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:20.963349 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:20.968083 (+ 4734us) server_negotiation.cc:407] Connection header received
0810 06:43:20.968836 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:20.968842 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:20.968928 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:20.968996 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:20.970037 (+ 1041us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52104: BlockingRecv error: recv got EOF from 127.0.0.1:52104 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:43:21.029515 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:21.002658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60370 (local address 127.31.59.253:36915)
0810 06:43:21.002873 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.002881 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.018889 (+ 16008us) server_negotiation.cc:407] Connection header received
0810 06:43:21.019191 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.019198 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.019419 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.019603 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:21.020446 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.021657 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.023823 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.024646 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.024752 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:21.025596 (+ 844us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:21.026284 (+ 688us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:21.028738 (+ 2454us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:21.029027 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 06:43:21.029142 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260810 06:43:21.051900 5957 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:21.047222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52112 (local address 127.31.59.252:40155)
0810 06:43:21.047406 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.047412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.050172 (+ 2760us) server_negotiation.cc:407] Connection header received
0810 06:43:21.050497 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.050503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.050640 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.050715 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:21.051676 (+ 961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52112: BlockingRecv error: recv got EOF from 127.0.0.1:52112 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:43:21.070600 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:21.063256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60374 (local address 127.31.59.253:36915)
0810 06:43:21.063459 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.063466 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.063763 (+ 297us) server_negotiation.cc:407] Connection header received
0810 06:43:21.063946 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.063951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.064069 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.064162 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:21.064686 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.065832 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.067842 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.068549 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.068669 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:21.069519 (+ 850us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:21.069694 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:21.069767 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:21.070107 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 06:43:21.070203 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260810 06:43:21.088181 5957 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:21.081999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52128 (local address 127.31.59.252:40155)
0810 06:43:21.082192 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.082198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.086676 (+ 4478us) server_negotiation.cc:407] Connection header received
0810 06:43:21.086848 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.086854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.086946 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.087051 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:21.087925 (+ 874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52128: BlockingRecv error: recv got EOF from 127.0.0.1:52128 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 06:43:21.107029 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:21.100116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60384 (local address 127.31.59.253:36915)
0810 06:43:21.100292 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.100299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.100792 (+ 493us) server_negotiation.cc:407] Connection header received
0810 06:43:21.100966 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.100980 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.101073 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.101170 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:21.101666 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.102789 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.104842 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.105536 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.105657 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:21.106166 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:21.106341 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:21.106425 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:21.106615 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 06:43:21.106696 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260810 06:43:21.115765 5957 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:21.113927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52134 (local address 127.31.59.252:40155)
0810 06:43:21.114113 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.114119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.114222 (+ 103us) server_negotiation.cc:407] Connection header received
0810 06:43:21.114551 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.114556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.114648 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.114757 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:21.115554 (+ 797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52134: BlockingRecv error: recv got EOF from 127.0.0.1:52134 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 06:43:21.130715 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:21.123685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60386 (local address 127.31.59.253:36915)
0810 06:43:21.123869 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.123876 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.124394 (+ 518us) server_negotiation.cc:407] Connection header received
0810 06:43:21.124665 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.124670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.124776 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.124871 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:21.125399 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.126566 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.128578 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.129269 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.129392 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:21.129926 (+ 534us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:21.130067 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:21.130166 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:21.130362 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 06:43:21.130442 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260810 06:43:21.139678 5957 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:21.137714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52138 (local address 127.31.59.252:40155)
0810 06:43:21.137952 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.137958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.137990 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:43:21.138188 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.138195 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.138333 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.138415 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:21.139440 (+ 1025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52138: BlockingRecv error: recv got EOF from 127.0.0.1:52138 (error 108)
Metrics: {"server-negotiator.queue_time_us":130}
I20260810 06:43:21.157949 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:21.151153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60398 (local address 127.31.59.253:36915)
0810 06:43:21.151332 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.151339 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.151895 (+ 556us) server_negotiation.cc:407] Connection header received
0810 06:43:21.152074 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.152083 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.152163 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.152261 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:21.152739 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.153869 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.155752 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.156457 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.156598 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:21.157124 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:21.157276 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:21.157374 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:21.157567 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 06:43:21.157649 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260810 06:43:21.167006 5957 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:21.164547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52152 (local address 127.31.59.252:40155)
0810 06:43:21.164740 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.164746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.165284 (+ 538us) server_negotiation.cc:407] Connection header received
0810 06:43:21.165575 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.165581 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.165669 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.165734 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:21.166785 (+ 1051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52152: BlockingRecv error: recv got EOF from 127.0.0.1:52152 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:43:21.194706 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:21.187096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60400 (local address 127.31.59.253:36915)
0810 06:43:21.187281 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.187287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.187853 (+ 566us) server_negotiation.cc:407] Connection header received
0810 06:43:21.188191 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.188198 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.188295 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.188367 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:21.189041 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.190330 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.192266 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.192992 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.193101 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:21.193832 (+ 731us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:21.194034 (+ 202us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:21.194133 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:21.194343 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 06:43:21.194440 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260810 06:43:21.203555 5957 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:21.201036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52154 (local address 127.31.59.252:40155)
0810 06:43:21.201261 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.201267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.201806 (+ 539us) server_negotiation.cc:407] Connection header received
0810 06:43:21.202096 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.202102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.202196 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.202268 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:21.203339 (+ 1071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52154: BlockingRecv error: recv got EOF from 127.0.0.1:52154 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 06:43:21.247051 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:21.239870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60416 (local address 127.31.59.253:36915)
0810 06:43:21.240108 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.240114 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.240627 (+ 513us) server_negotiation.cc:407] Connection header received
0810 06:43:21.240818 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.240825 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.240925 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.241040 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:21.241524 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.242746 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.244710 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.245456 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.245627 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:21.246082 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:21.246234 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:21.246384 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:21.246532 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 06:43:21.246628 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 06:43:21.258590 5957 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:21.256050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52162 (local address 127.31.59.252:40155)
0810 06:43:21.256240 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.256246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.256768 (+ 522us) server_negotiation.cc:407] Connection header received
0810 06:43:21.257049 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.257055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.257205 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.257280 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:21.258354 (+ 1074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52162: BlockingRecv error: recv got EOF from 127.0.0.1:52162 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:43:21.333199 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:21.326902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60424 (local address 127.31.59.253:36915)
0810 06:43:21.327145 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.327152 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.327606 (+ 454us) server_negotiation.cc:407] Connection header received
0810 06:43:21.327863 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.327869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.327947 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.328025 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:21.328599 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.329328 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.331408 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.331819 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.331918 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:21.332428 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:21.332549 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:21.332625 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:21.332821 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 06:43:21.332904 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260810 06:43:21.341434 5957 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:21.339353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52166 (local address 127.31.59.252:40155)
0810 06:43:21.339570 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.339577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.339990 (+ 413us) server_negotiation.cc:407] Connection header received
0810 06:43:21.340176 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.340180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.340251 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.340330 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:21.341204 (+ 874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52166: BlockingRecv error: recv got EOF from 127.0.0.1:52166 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:43:21.477409 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:21.472948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60432 (local address 127.31.59.253:36915)
0810 06:43:21.473176 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.473183 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.473418 (+ 235us) server_negotiation.cc:407] Connection header received
0810 06:43:21.473575 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.473580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.473651 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.473728 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:21.474047 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.474745 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.475956 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.476353 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.476441 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:21.476827 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:21.476928 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:21.476994 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:21.477100 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 06:43:21.477161 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260810 06:43:21.484360 5957 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:21.482631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52178 (local address 127.31.59.252:40155)
0810 06:43:21.482838 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.482843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.483123 (+ 280us) server_negotiation.cc:407] Connection header received
0810 06:43:21.483289 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.483293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.483377 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.483456 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:21.484158 (+ 702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52178: BlockingRecv error: recv got EOF from 127.0.0.1:52178 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 06:43:21.502900 5975 catalog_manager.cc:1259] Loaded cluster ID: fcde5cef235e40ceb910bb6dec86a15a
I20260810 06:43:21.502982 5975 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b: loading cluster ID for follower catalog manager: success
I20260810 06:43:21.505390 5975 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b: acquiring CA information for follower catalog manager: success
I20260810 06:43:21.507144 5975 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7a3d7b807f434f3e872572eee52ed96b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:43:21.752717 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:21.747860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60434 (local address 127.31.59.253:36915)
0810 06:43:21.748063 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.748070 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.748247 (+ 177us) server_negotiation.cc:407] Connection header received
0810 06:43:21.748459 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.748463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.748534 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.748613 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:21.749026 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.749748 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.751113 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.751508 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.751620 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:21.752015 (+ 395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:21.752134 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:21.752211 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:21.752397 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 06:43:21.752476 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:43:21.763247 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:21.757795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52182 (local address 127.31.59.252:40155)
0810 06:43:21.758122 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.758127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.758270 (+ 143us) server_negotiation.cc:407] Connection header received
0810 06:43:21.758501 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.758506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.758681 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.758796 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:21.759138 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.759865 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.761091 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.761537 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.761663 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:21.762097 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:21.762703 (+ 606us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:21.762816 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:21.762903 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 06:43:21.762964 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 06:43:21.775996 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:21.768732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60442 (local address 127.31.59.253:36915)
0810 06:43:21.768895 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.768901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.769732 (+ 831us) server_negotiation.cc:407] Connection header received
0810 06:43:21.770014 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.770020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.770110 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.770202 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:21.770742 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.771861 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.773945 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.774609 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.774726 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:21.775264 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:21.775399 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:21.775487 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:21.775602 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 06:43:21.775682 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:43:21.782373 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:21.768757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52186 (local address 127.31.59.252:40155)
0810 06:43:21.768924 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.768929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.775987 (+ 7058us) server_negotiation.cc:407] Connection header received
0810 06:43:21.776431 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.776437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.776518 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.776575 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:21.777100 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.778165 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.780262 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.780920 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.781049 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:21.781623 (+ 574us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:21.781749 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:21.781834 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:21.781982 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 06:43:21.782077 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:43:21.787174 6242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:21.768466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45960 (local address 127.31.59.254:33309)
0810 06:43:21.769045 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:21.769051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:21.781866 (+ 12815us) server_negotiation.cc:407] Connection header received
0810 06:43:21.782073 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:21.782078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:21.782331 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:21.782525 (+ 194us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:21.783148 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.783944 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.785247 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:21.785689 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:21.785802 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:21.786173 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:21.786554 (+ 381us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:21.786646 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:21.786754 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 06:43:21.786848 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":431,"threads_started":1}
I20260810 06:43:21.791532 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 5580
I20260810 06:43:21.792513 5688 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:21.899710 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 5580
W20260810 06:43:21.926967 5869 connection.cc:535] server connection from 127.0.0.1:52064 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 06:43:21.927426 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 5707
I20260810 06:43:21.928457 5815 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:22.033941 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 5707
I20260810 06:43:22.057771 31983 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu pid 5838
I20260810 06:43:22.058859 5943 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:22.162899 31983 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq68xXZ/build/asan/bin/kudu with pid 5838
2026-08-10T06:43:22Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4712 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9873 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (370338 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST